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PREFACE 


This  paper  presents  guidelines  for  tailoring  the  DOD-STD-2167A  Software  Development 
Standard  to  software  that  may  be  developed  for  the  Strategic  Defense  Initiative  Organization 
(SDIO).  It  is  based  on  a  preliminary  examination  of  how  DOD-STD-2167A  might  apply  to 
Strategic  Defense  System  (SDS)  software  in  various  stages  of  research  and  development.  It 
provides  general  guidelines  for  use  by  SDIO,  agents,  and  contractors. 

The  document,  developed  under  Task  Order  T-R5-422,  is  part  of  IDA’s  effort  to  support  the 
SDIO  by  providing  analyses  and  recommendations  for  software  activities.  The  document  was 
reviewed  on  16  July  1987  by  the  members  of  the  following  CSED  Peer  Review:  Audrey 
Hook,  Jack  Kramer,  Catherine  McDonald,  Sarah  Nash,  and  Robert  Winner.  In  addition,  the  draft 
version  was  externally  reviewed  by  and  comments  were  received  from  members  of  the  SDIO 
community. 
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Guidelines  for  Tailoring  DOD-STD-2167A 
for  SDI  Related  Software  Development 


1.  PURPOSE 

This  paper  provides  the  SDIO  with  a  recommended  tailoring  of  DOD- STD-2 1 67A  and  related 
Data  Item  Descriptions  (DIDs),  including  a  suggested  interpretation  of  the  review  requirements 
in  MIL-STD-1521B  (Technical  Reviews  and  Audits  for  Systems,  Equipments,  and  Computer 
Software),  for  use  as  guidelines  in  developing  prototype  and  production  software.  Although 
s>^ill  a  draft.  Revision  A  of  the  Standard  was  chosen  as  the  subject  of  this  paper  since  it  will 
supercede  the  current  standard  in  1987  and  because  it  more  adequately  addresses  modern 
software  technology  ideas.  This  paper  does  not  suggest  that  changes  be  made  to  the  Standard, 
but  rather  should  be  viewed  as  an  interpretation  of  the  Standard.  However,  based  in  part  on 
recommendations  derived  from  the  initial  draft  of  this  paper  and  other  work  at  IDA,  some 
changes  in  the  Standard  have  been  made.  The  guidelines  in  this  paper  are  intended  to  be 
reflected  in  Requests  for  Proposals  (RFPs)  distributed  to  vendors  proposing  to  develop  SDI 
software  and  in  contracts  let  for  SDI  software  acquisition. 


2.  SCOPE 

Section  3.  of  this  paper  outlines  the  software  needs  of  the  SDI  and  describes  how  those  needs 
may  not  be  fulfilled  with  an  untailored  DOD-STD-2167A.  (For  reference,  a  copy  of  the 
Standard  is  found  in  .Appendi.x  D.)  The  specific  tailoring  recommendations  for  IX)D-STD-2]67.A. 
are  found  in  Section  4.  and  the  applicability  of  MIL-STD-1521B  reviews  over  the  range  of 
potential  SDI  software  programs  is  discussed.  Appendix  B  covers  the  recommendations  for 
tailoring  the  associated  DIDs.  Some  notes  and  disclaimers  are  in  Section  5. 


3.  BACKGROUND 


3.1  SDI  Software  Needs 

The  software  requirements  of  SDI  Battle  Management/Command,  Control,  and  Communications 
(BM/C3)  stretch  beyond  the  current  state-of-the-practice  in  performance,  capacity,  and 
adaptability.  To  meet  those  requirements  in  the  future,  software  design  and  implementation 
will  need  to  be  innovative  and  must  evolve  rapidly  with  the  changing  edge  of  technology. 
Ideas  will  have  to  be  translated  quickly  into  .software  prototypes  to  demonstrate  the  validity 
of  the  technology  they  represent.  In  addition,  libraries  of  certified'  software  components 
produced  acquired  for  SDI  or  other  Government  funded  programs  will  be  heavily  used. 

.■\11  aspects  of  software,  from  early-on  studies  of  theoretical  concepts  expressed  formally  as 


Certified  in  this  ca.se  refers  to  software  components  guaranteed  to  be  free  of  Trojan  Horses’ 
or  other  hostile  entities  and  which  are  functionallv  correct  to  their  specifications. 
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abstract  equations,  represented  as  finite  state  automata,  etc.,  through  to  the  development,  testing, 
and  deployment  of  entire  systems,  are  actively  being  pursued  today  by  the  computing 
community  for  use  in  a  wide  range  of  applications.  Although  interested  in  recognizing  and 
supporting  emerging  technologies,  the  SDIO  may  only  fund  software  efforts  that  result  in 
products  expressed  in  executable  software,  which  are  applicable  to  and/or  extensible  into 
Stategic  Defense  System  (SDS)  subsystems.  Ideally,  but  not  exclusively,  the  software  developed 
for  one  phase  of  the  program  will  be  upwardly  compatible  with  succeeding  phases.  That  is, 
the  final  software  product  from  one  phase  becomes  the  initial  specifications  for  a  succeeding 
phase.  This  differs  from  a  more  traditional  approach  of  funding  programs  that  do  not  rely  on 
or  do  not  take  advantage  of  prior  products  as  an  initial  baseline.  B.M/C3  potential  software 
programs  will  fall  into  four  general  phases: 


1.  Concept  Formulation  (CF)  -  where  software  is  developed  to  asssist  in  proving 
concepts,  defining  technology  requirements,  or  for  demonstrating  the  feasibility  or 
appropriateness  of  technical  approaches. 

2.  Initial  Prototypes  (IP)  -  where  software  is  developed  to  provide  an  initial 
demonstration  of  components  in  a  system  context  to  assist  in  defining  interfaces 
and  in  refining  functional  requirements. 

3.  .\dvanced  Prototypes  (AP)  -  where  systematic,  incrementally  advanced,  integratable 
software  implementations  of  the  initial  prototypes  provide  detailed  functional  and 
performance  requirements  for  system  development. 

4.  Full-Scale  Development  (FSD)  -  where  complete  operational  code,  evaluated  against 
specific  full-scale  engineering  decision  (FSED)  criteria,  is  developed  for  real-world 
applications. 

The  SDIO  classifies  programs  in  the  CF  and  IP  categories  as  “technology”  programs,  while 
those  in  AP  (and  possibly  a  small  number  in  IP)  are  classified  as  “experimental  validation” 
programs.  The  important  point  in  the  classification  scheme  is  not,  however,  the  relationship 
of  the  categories  to  a  funding  classification,  but  rather  the  uses  to  which  the  results  of  a 
program  are  to  be  applied.  As  a  consequence,  the  requirements  and  the  applicability  of 
military  standards  will  vary  among  the  four  categories. 


з. 2  DOD-STD-2167A 

The  Defense  System  Software  Development  Standard.  DOD-STD-2167A  (27  October  1987  draft), 
contains  the  requirements  for  the  development  of  mission-critical  computer  resources  (MCCR) 
software.  DOD-STD-2167.A  is  intended  to  establish  “uniform  requirements  for  software 
development  that  are  applicable  throughout  the  system  life  cycle.”  Although  the  Standard  is 
intended  for  use  through  the  entire  life  cycle  and  “is  not  intended  to  specify  or  discourage  the 

и. se  of  any  particular  software  development  method”,  it  must  be  tailored  to  suit  the  specific 

contract  needs  of  individual  programs.  In  general,  the  Standard  provides  an  adequate 

mechanism  for  the  software  development  process.  Some  parts  however,  require  a  clarification, 

additional  emphasis,  or  a  specific  interpretation  to  support  the  software  philosophies  held  bv 
the  SDIO. 

FSD  software  will  make  heavy  use  of  reusable  modules  from  non-developmental  software 

(NDS)  libraries  of  Government  furnished  software  (GFS)  components  or  certified  Commercial 
Off-The  Shelf  (COTS)  software.  Procedures  for  developing,  cataloging,  storing,  accessing,  and 
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controlling  certified  software  components  in  reusable  component  libraries  will  be  very  complex. 
A  hierarchy  of  attributes  relating  to  the  security  certification,  fault  tolerance,  and  verifiability 
of  each  library  entry  will  have  to  be  established  and  maintained  through  an  intelligent 
library  management  system  to  make  effective  reuse  of  software.  A  straight-forward  top-down 
design  approach  would  not  naturally  lend  itself  to  constructing  software  from  these  reusable 
modules.  The  reuse  of  software  modules  often  requires  an  element  of  the  bottom-up  approach 
to  design.  The  Standard  supports  reuse  by  requiring  the  contractor  to  consiaer  using  NDS 
software  (para  4.2.4).  However,  the  specific  requirement  to  exploite  reusable  software  is  not 
emphasized.  Tailoring  is  needed  to  require  development  methodologies  that  apply  or  encourage 
this  practice. 

Similarly,  using  the  classic  waterfall  life  cycle  methodology  within  research  activities  is 
constraining.  The  waterfall  method  is  an  unambiguous  method  for  implementing  production 
software,  beginning  with  requirements  and  proceeding  linearly  through  specifications,  design, 
implementation,  and  maintenance.  With  this  method  however,  errors  in  requirements, 
specification,  and  design  are  not  detected  until  the  software  is  implemented.  Finding  errors  so 
late  in  the  life  cycle  adversely  impacts  not  only  cost  but  also  delivery  schedules.  Errors  in 
early  phases  are  especially  common  in  development  projects  for  systems  that  have  not  been 
previously  implemented  and  for  which  the  requirements  are  not  well-defined  or  understood.  To 
avoid  this  pitfall,  the  SDIO  is  employing  a  prototyping  discipline  whereby  key  portions  of  the 
system  are  developed  to  ensure  the  correctness  of  the  system  requirements  and  specifications. 
Errors  detected  in  the  prototyping  efforts  result  in  early  corrections  to  the  system  specification, 
saving  time  and  helping  to  keep  costs  within  budget.  The  Standard  supports  the  use  of 
prototyping  by  specifically  requiring  that  the  “software  development  process  shall  include  the 
following  major  activities,  which  may  overlap  and  may  be  applied  iteratively  or  recursively...” 
(para  4.1.1).  Also,  the  contractor  is  required  to  use  a  systematic  and  well-documented  software 
development  method  (para  4.2.1).  Protoyping  is  such  a  method.  However,  there  is  no  explicit 
requirement  for  prototyping.  The  SDIO  promotion  of  a  prototyping  development  methodology 
needs  to  be  refelected  in  the  tailorings. 

The  SDI  software  will  be  developed  by  a  variety  of  organizations  ranging  from  universities  to 
aerospace  firms.  The  application  of  EX3D-STD-2167A’s  documentation  requirements  (para  4.6.4 
and  Figure  2.)  needs  to  be  tailored  appropriately.  For  example,  the  impact  of  a  CF  activity 
conducting  technology  feasibility  studies  might  become  reduced  if  the  activity  is  mired  in 
formal  documentation.  It  is  reasonable  that  a  minimum  of  formal  documentation  may  be  all 
that  is  needed  with  an  IP  prototype.  On  the  other  hand,  an  FSD  activity  may  not  only  be 
constructing  an  operational  product,  as  opposed  to  an  experimental  prototype,  but  would  have 
well-defined  requirements  and  specifications  to  use  as  the  required  formal  documentation. 
Along  the  same  lines,  the  formal  reviews  and  audits  as  specified  in  MIL-STD-1521B  are  less 
applicable  for  CF  and  IP  activities.  The  Standard  supports  this  view  by  allowing  the  degree  to 
which  formal  reviews  are  required  to  be  determined  by  the  contract  (para  4.1.2).  This  is 
relatively  explicit  and  should  not  need  further  clarification  through  tailoring.  However,  some 
suggested  documentation  and  review  requirements  for  various  activities  are  shown  in  Section 
4.4. 

Finally,  the  Standard  does  not  explicitly  require  the  use  of  Ada'  and  Ada  program  design 
language  (PDL)  as  stated  in  DoD  Directives  3405.1  and  3405.2.  Even  if  a  waiver  for  use  of  a 
language  other  than  Ada  will  be  requested,  the  Standard  needs  to  be  tailored  to  reflect 
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conformity  with  DoDDs  3405.1  and  3405.2.  In  addition,  the  use  of  SDI  Ada/Process  Description 
Language  (SA/TDL)  to  describe  the  design  of  information  processing  elements  of  the  SDS  will 
be  required. 


3.3  Related  Efforts 

In  addition  to  applying  military  standards  to  the  software  development  process  as  a  whole, 
other  efforts,  such  as  for  testing  and  evaluation,  are  underway  to  “standardize”  specific  aspects 
of  software  development. 

A  Test  and  Evaluation  Master  Plan  (TEMP)  is  being  developed  to  provide  guidance  for  the 
planning,  execution,  and  reporting  of  BM/C3  software  test  and  evaluation  (T&E)  activities.  This 
TEMP  will  differ  from  other  TEMPs  for  large  and  complex  software  systems,  such  as  the 
World  Wide  Military  Command  and  Control  Information  System  (WIS),  in  that  this  TEMP 
identifies  the  need  for  formal  notations  to  support  the  rigorous  specification  of  testing 
requirements  as  part  of  software  requirements  and  specifications. 

Since  any  system  cannot  be  tested  except  against  its  specifications,  T&E  will  be  facilitated  by 
using  a  formal  notation  that  provides  a  grammar  for  ensuring  complete  specification  of  critical 
software  characteristics,  their  threshold  values,  and  associated  testing  requirements. 


4.  RECOMMENDED  TAILORINGS 

Tailoring  is  a  cost-effective  method  for  applying  the  requirements  to  meet  the  specific  needs  of 
particular  programs.  .4  well-tailored  standard  will  avoid  unnecessary  activities  during  the 
acquisition  and  will  thereby  reduce  the  cost  of  the  overall  software  development  effort. 

The  tailoring  of  the  Standard  involves  three  general  steps:  tailoring  the  General  Requirements 
in  Section  4.,  tailoring  the  Specific  Requirements  in  Section  5.,  and  finally  tailoring  the 
associated  DIDs.  To  facilitate  an  understanding  of  some  of  the  tailorings,  additional  definitions 
have  been  suggested  for  Section  3.  All  paragraph  or  figure  references  that  follow  are  from  the 
Standard  or  DIDs.  All  changes  within  a  paragraph  will  be  denoted  with  change  bars  and 
highlighted  with  boldface  type. 


4.1  Additional  Definitions 
Additions  to  Section  3.  DEFINITION'S. 

I3.XX  Development  phases: 

I3.XX.1  Concept  Formulation.  Development  of  software  to  eissist  in  proving  concepts, 
defining  technology  requirements,  or  demonstrating  the  feasibility  or  appropriateness 
of  technical  approaches. 

I3.XX.2  Initial  Prototypes.  Development  of  software  to  provide  an  initial 
demonstration  of  components  in  a  system  context  to  assist  in  defining  interfaces  and 
in  refining  functional  requirements. 

I3.XX.3  Advanced  Prototypes.  Developiuent  of  systematic,  incrementally  advanced, 
integratable  software  implementations  of  initial  prototypes  to  provide  detailed 
functional  and  performance  requirements  for  system  development. 

I3.XX.4  Full-Scale  Development.  Development  of  operational  software  systems. 
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I  undertaken  based  on  an  evaluation  of  prototype  and  study  results  with  respect  to 
specific  full-scale  engineering  decision  criteria. 


4.2  General  Requirements  Tailoring 
Changes  in  Section  4.  GENERAL  REQUIREMENTS. 

para  4.1.1  Software  development  process.  ...  The  software  development  process  shall  include  the 

(following  major  activities,  as  applicable  to  the  development  phase  (see  Figures  2a-2d) 
and  as  specified  in  the  contract: 

I  a.  Preliminary  Software  Requirements  Analysis 
I  b.  Preliminary  Design  (Initial  Prototype) 

I  c.  Iterative  Prototype  Design  Change/Refinement/Expansion 
I  d.  Refined  Software  Requirements  Analysis 

e.  Detailed  Design 

f.  Coding  and  Unit  Testing 

g.  CSC  Integration  and  Testing 

h.  CSCI  Testing 
The  last  item  “i”,  is  deleted. 


I  para  4.1.2  Formal  reviews/audits.  ™  Figures  2a  through  2d  illustrate  the  occurrence  of 
I  formal  reviews  and  audits  for  the  four  software  development  phases  and  shows  the 
I  relationship  of  the  minimal  set  of  deliverable  products  to  baselines  and  the  Developmental 
Configuration. 


para  4.1.9  Software  development  library.  ...  The  contractor  shall  document  and  implement 
procedures  for  controlling  software  and  associated  documentation  residing  within  the  SDL.  The 

contractor  shall  also  document  and  implement  procedures  for  cataloging,  storing, 
accessing,  and  controlling  reusable  software.  The  contractor  shall  maintain  the  SDL  ... 


I  para  4.2.4  Non-developmental  software.  The  contractor  shall,  to  the  maximum  extent 

(possible,  incorporate  non-developmental  software  (N'DS),  such  as  reusable  software  that  was 
internally  developed  or  obtainable  as  Government  furnished  software  (GFS),  into  the 

(deliverable  software.  The  contractor  shall  document  plans  for  using  NDS.  NDS  shall  be 
catalogued,  stored,  and  controlled  through  the  procedures  for  the  software  devlopment 
library,  NDS  may  be  ... 
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I  para  4.2.7  High  order  language.  The  contractor  shall  use  the  High  Order  Language  (HOL)  Ada, 
as  required  by  DoD  Directives  3405.1  and  3405.2,  to  code  deliverable  software. 

The  last  sentence  in  para  4.2.7,  “If  no  HOL  is  deleted. 


I  para  4.2.8  Design  and  coding  standards.  The  contractor  shall  document  and  implement  the  Ada 
PDL  and  Ada  coding  standards  to  be  used  in  the  development  of  deliverable  software.  In 
addition,  the  contractor  shall  use  SDI  Ada  Process  Description  Language  to  provide  a 
formal  description  of  the  design  for  simulation  and  evaluation  prior  to  the 
development  of  code.  Software  coding  standards  shall  comply  with  the  requirements  specified 
in  Appendix  B. 


4.3  Detailed  Requirements  Tailoring 

Changes  in  Section  5.  DETAILED  REQUIREMENTS. 
No  changes. 


4.4  Data  Item  Descriptions 

The  DIDs  associated  with  DOD-STD-2167A  describe  a  concise  and  complete  set  of  documents 
for  recording  information  required  by  the  Standard.  As  with  the  Standard  itself,  the  DIDs 
may  be  tailored.  The  Contract  Data  Requirements  List  (CDRL),  DD  Form  1423,  incorporated 
into  a  contract  identifies  the  data  requirements  that  shall  be  developed  as  specified  by 
approved  DIDs.  Not  every  DID  associated  with  the  Standard  is  applicable  to  every  contract 
however.  Appendix  B  in  this  paper  provides  a  tailoring  of  those  DIDs  requiring  modification 
to  maintain  consistency  with  the  Standard  as  tailored  by  this  paper.  Applicability,  tailoring, 
etc.  of  the  other  DIDs  is  contract  dependent.  In  addition,  there  are  items  for  which  DIDs  do 
not  exist,  e.g.  the  Program  Report  for  CF  activities.  Preparation  of  draft  DIDs  for  these  items 
may  be  considered  as  a  future  effort. 


4.5  Documentation  and  Review  Requirements  (M1L-STD-1521B) 

The  SDIO  will  potentially  fund  programs  at  all  four  phases  of  development.  As  stated  before, 
these  programs  are  varying  in  their  scope  of  activities  and  deliverable  products.  Therefore,  it  is 
reasonable  that  the  documentation  and  review  requirements  across  these  four  phases  will  also 
vary.  Fundamental  research  performing  exploratory  development  of  experimental  prototypes 
may  not  reach  the  stage  of  detailed  design,  while  full-scale  engineering  development  programs 
will  develop  completely  integrated,  operationally  tested  products. 

Specific  requirements  for  documentation  and  reviews  are  individually  contract  dependent  and 
the  guidelines  for  tailoring  MIL-STD-1521B  are  found  in  Appendix  J  of  that  standard.  The 
Standard  is  very  clear  in  paragraph  100.4.1,  ‘The  key  to  tailoring  MIL-SI  !>  1521  is  to  match 
the  MIL-STD-1521  requirements  against  the  details  of  the  applicable  SOW/Contractual  task 
requirements.”  The  following  though,  may  serve  as  a  possible  generic  solution  or  as  a  baseline 
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for  BM/C3  software  programs.  The  mapping  of  the  documentation  and  reviews  listed  for  the 
IP  and  AP  phases  into  the  DIDs  and  .MIL-STD'1521B  is  shown  in  parentheses. 

4.5.1  Concept  Formulation 

The  requirements  for  CF  phase  programs  are  relatively  simple.  Developing  software  to  prove 
concepts  or  for  defining  requirements  is  not  aimed  at  producing  an  operational  implementation 
but  rather  at  developing  technology  into  a  feasible  demonstration  with  minimal  risks.  None  of 
the  documents  required  by  DOD-STD-2167A  nor  the  reviews  required  by  MIL-STD-1521B  are 
applicable.  However,  all  SDIO  funded  activities  are  expected  to  generate  products,  either  in  the 
form  of  documents,  code,  or  both.  For  CF  programs  the  following  will  be  required: 


Documentation: 

•  Program  Report  consisting  of  textual  documentation  describing: 

-  Capabilities  and  functionality  of  the  technology  developed 

-  Interface  with  other  SDI  activities 

-  Benefits  provided  to  SDI 

-  Approach  used  to  develop  this  technology 

-  Test  results  from  simulating  the  prototype 

•  Concepts  Prototype  Code,  either  as  Ada,  SA/PDL,  or  a  specifically  approved 
language 


Reviews: 

•  Concepts  Prototype  Product  Review 

4.5.2  Initial  Prototypes 

IP  programs  will  develop  the  initial  prototypes  in  SA/PDL,  often  using  the  technology 
demonstrated  in  CF  programs.  Preliminary  requirements  will  be  derived  directly  from  CF  code. 
Test  plans  and  procedures  will  also  be  developed;  testing  will  be  conducted  and  reported.  An 
IP  program  would  need  the  following  documents  and  formal  reviews: 


Documentation: 

•  Initial  Prototype  Requirements  Specification  (Preliminary  Software  Requirements 
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Specification)^ 

♦  Initial  Prototype  Development  Plan  {PreUminary  Software  Development  Plan) 

*  Initial  Prototype  Design  Document  {Preliminary  Software  Top-Level  Design 
Document) 

*  Initial  Prototype  Source  and  Object  code 

•  Initial  Prototype  Test  Description  {Preliminary  Software  Test  Description) 

•  Initial  Prototype  Test  Report  {Preliminary  Software  Test  Report) 

Reviews: 

•  Initial  Prototype  Specification  Review  {Prelijtdnary  Software  Specification  Review) 

•  Initial  Prototype  Design  Review  (Preliminary  Design  Review) 

*  Initial  Prototype  Product  Review 

4-53  Advanced  Prototypes 

Programs  in  the  AP  phase  will  involve  systematic,  incremental  growth  of  IP  prototypes  and 
will  result  in  very  nearly  the  implementation  of  a  complete  system  or  integratable  component 
capable  of  handling  real-world  operational  constraints  in  the  simulation  system.  The  system 
design  will  be  taken  directly  from  the  SA/PDL  of  the  preceeding  IP.  Structural,  functional, 
and  interface  organization  and  procedures  will  be  documented  in  a  software  development  plan. 
As  the  incremental  prototypes  are  produced,  tested,  and  refined,  the  final  requirements  and  the 
final  design  documents  will  be  produced.  Test  plans  and  procedures  will  also  be  produced; 
testing  will  be  conducted  and  reported.  An  AP  program  should  include  the  following 
documentation  and  reviews; 


Documentation: 

•  Software  Prototype  Development  Plan  {Refined  Software  Development  Plan) 

•  Software  Prototype  Requirements  Specifications  {Refined  Software  Requirements 
Specification) 

•  Software  Prototype  Top-Level  Design  Documents  {Refined  Software  Top-Leve' 
Design  Document) 

•  Software  Prototype  Detailed  Design  Document  (Software  Detailed  Design  Document) 


’Items  in  ‘t  )”  are  the  relevant  DOD-STD-2167A  DIDs 
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•  Software  Prototype  Source  and  Object  code  j 

•  Software  Prototype  Test  Description  {Refined  Software  Test  Description)  I 

( 

•  Software  Prototype  Test  Report  {Refined  Software  Test  Report)  i 

I 

I 

I 

Reviews: 


•  Final  Software  Prototype  Specification  Review  {Refined  Software  Specification 
Review) 

•  Final  Software  Prototype  Top-Level  Design  Review  {Refined  Preliminary  Design 
Review) 

•  Final  Software  Prototype  Design  Review  (Critical  Design  Review) 

•  Final  Software  Prototype  Test  Readiness  Review  (Test  Readiness  Review) 

•  Final  Software  Prototype  Product  Review 


4^.4  Full-Scale  Development 

A  full-scale  engineering  development  program  will  result  in  a  completely  operational,  fully 
integrated,  and  thoroughly  tested  software  system.  The  software  requirements  and  detailed 
design  will  have  been  refined  from  AP  programs.  From  this  point,  depending  on  the  contract, 
it  is  reasonable  to  produce  all  of  the  documentation  and  prepare  for  all  of  the  formal  reviews 
specified  in  DOD-STD-2167A  and  MIL-STD-1521B  by  using  the  waterfall  life  cycle  method  as 
a  model. 


5.  CONCLUSIONS 

The  27  October  1987  draft  of  IX)D-STD-2167A  and  01  May  1987  drafts  of  the  associated 
DIDs  were  used  in  this  paper.  This  version  of  DOD-STD-2167A  has  been  forwarded  by  the 
Joint  Logistical  Commanders  Joint  Policy  Coordinating  Group  on  Resource  Management  to  the 
Defense  Product  Standards  Office  for  release.  Any  changes  made  to  the  Standard  prior  to  this 
release  should  not  affect  this  paper. 

As  mentioned  in  the  previous  section,  the  requirements  for  documentation  and  reviews  are 
contract  specific.  The  SDIO  and/or  the  contracting  agency  should  work  with  the  contractor  to 
correctly  specify  the  appropriate  documents  and  reviews  needed. 

The  Sundard,  although  tuned  to  more  modern  methods  of  software  development,  must  still  be 
tailored  to  provide  a  structure  for  cost-effective  software  acquisition.  The  recommendations  in 
this  paper  are  untried.  However,  a  good  example,  along  the  same  lines,  of  a  completely 
tailored  Standard  is  currently  in  draft  form  out  of  the  System  Integration  Office  of  the  U.S. 
Space  Command. 
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APPENDIX  A 
LIST  OF  ACRONYMS 


ABM  Anti-Ballistic  Missle 

AP  Advanced  Prototypes 

BM/C3  Battle  Management/Command,  Control, 

Communications 

CDRL  Contract  Data  Requirements  List 

CF  Concept  Formulation 

COTS  Commercial  Off-the-Shelf 

CSC  Computer  Software  Component 

CSCl  Computer  Software  Configuration  Item 

DID  Data  Item  Description 

DOD  Department  of  Defense 

FSD  Full-Scale  Development 

FSED  Full-Scale  Engineering  Decision 

GFS  Goverment  Furnished  Software 

HOL  High  Order  Language 

IP  Initial  Prototypes 

LLCSC  Lower-Level  Computer  Software  Component 

MCCR  Mission-Critical  Computer  Resources 

NDS  Non-Developmental  Software 

PDL  Program  Design  Language 

RFP  Request  for  Proposals 

SA/PDL  SDl  Ada/Process  Description  Language 

SDI  Strategic  Defense  Initiative 

SDIO  Strategic  Defense  Initiative  Organization 

SDL  Software  Development  Library 

SDP  Software  Development  Plan 

SDS  Strategic  Defense  System 

SOW  Statement  of  Work 

TLCSC  Top-Level  Computer  Software  Component 

W'IS  W'orld  Wide  Military  Command  and  Control  Information  System 
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APPENDIX  B 

Applicable  Data  Item  Description  Tailorings 

B.l  Software  Development  Plan  (DI-MCCR-80030A) 

Changes  in  Section  10.  PREPARATION  INSTRL'CTIO.NS 

para  10.2.5.9  Software  development  library.  This  paragraph  shall  be  numbered  3.9  and  shall 
describe  the  software  development  library  (SDL)  to  be  used  by  the  contractor  for  controlling 
the  software  and  associated  documentation.  This  paragraph  shall  include  a  description  of  the 
contractor’s  procedures  and  methods  for  establishing  and  implementing  the  SDL,  the  contractor’s 
access  and  control  procedures  for  data  stored  in  the  SDL,  and  the  contractor's  procedures 
for  cataloging,  storing,  and  accessing  reusable  component  software. 


para  10.2.6.2.1  Software  development  techniques  and  methodologies.  This  subparagraph 
shall  be  numbered  4.2.1  and  identify  and  describe  the  techniques  and  methodologies  the 
contractor  plans  to  use  to  perform: 

I  a.  Preliminary  Software  Requirements  Analysis  (including  structured  requirements 
analysis  tools,  techniques,  or  a  combination  of  both) 

I  b.  Preliminary  Design  (Initial  Prototype) 

I  c.  Iterative  Prototype  Design  Change/Refinement/Expansion 
I  d.  Refined  Software  Requirements  Analysis 

e.  Detailed  Design 

f.  Coding  and  Unit  Testing 

g.  CSC  Integration  and  Testing 

h.  CSCI  Testing 


I  para  10.2.6.2.4  Design  standards.  This  subparagraph  shall  be  numbered  4.2.4  and  shall 
reference  the  Ada  PDL  design  standards  the  contractor  will  use  in  developing  the 
I  software.  This  subparagraph  shall  also  reference  the  SA/PDL  used  for  process 
description. 

The  last  sentence  in  para  10.2.6.2.4,  “If  the  contractor  ...”  is  deleted. 


I  para  10.2.6.2.5  Coding  standards.  This  subparagraph  shall  be  numbered  4.2.5  and  shall 
reference  the  Ada  coding  standards  in  the  contractor  will  use  in  developing  the 
software.  Any  planned  modifications  to  the  referenced  coding  standards  shall  be  documented  in 
this  subparagraph. 

The  second  sentence  in  para  10.2.6.2.5,  “If  the  contractor  ..”  is  deleted. 
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B.2  Software  Requirements  Specification  (DI-MCCR-80025A) 

para  10.2.5.3.1  Programming  language(s).  This  subparagraph  shall  be  numbered  3.3.1  and 
I  shall  reference  the  Ada  programming  language  required  by  DoDDs  3405.1  and  3405.2. 

to  be  used  to  implement  the  CSCI. 

para  10.2.5.3.2  Coding  standards.  This  subparagraph  shall  be  numbered  3.3.2  and  shall  specify 
I  directly  or  by  reference  the  Ada  coding  standards  under  which  the  CSCI  shall  be 
implemented. 

para  10.2.5.3.3  Compiler/assembler.  This  subparagraph  shall  be  numbered  3.3.3  and  shall 
I  specify  the  Ada  compiler  to  be  used  to  translate  the  CSCI  implementation. 

para  10.2.5.4.1  Design  standards.  This  subparagraph  shall  be  numbered  3.4.1  and  shall  specify 
I  directly  or  by  reference  the  Ada  PDL  standards  under  which  the  CSCI  shall  be  implemented. 


B.3  Software  Product  Specification  (DI-MCCR-80029A) 

I  para  Compiler/assembler.  This  paragraph  shall  be  numbered  3.5  and  shall  specify  the  Ada 
compiler  used  to  translate  the  source  code. 
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1.  SCOPE 


1 1  V»  puipoM  of  this  stsndsfd  is  to  sstsbilsh  rsquirsmsnts  to  bs  applied  during  the 

^■jTifS^dewefcccneiit.  snd  support  of  software  systems. 

1.J  Thi.  »ttnd.nl  WPIM  to  tho  lo  th.  oonwet  th.  St«.m.nt  of 

Wolft  (SOW),  and  the  Contract  Data  Requirements  List  (CORL). 

,.l,  H».looo,.nt.  App«e«lon  ot  m.  Uod«d  o««»  b.  coo«fln«.d  with  MIL-STO499. 

Engineering  Management,  for  total  system  development 

1  2,  au«iHv nroarafti.  Application  o#  this  Standard  must  be  coordinated  with  DOD-STD-2168. 

Defense  System  Software  Quality  Program,  to  provide  a  complete  software  quality  program. 

1.23  Pirniware.  This  standard  applies  to  tho  development  or  support  of  the  software  element  of 
firmware.  This  standard  does  not  apply  to  the  development  o#  the  hardware  element  ot  flonware. 

1  o  a  rtAdwara  deveiooed  hv  Government  aoencies.  The  provisions  of  this  standard  may  be  applied  to 

d««lopm««  « 

•ooordanee  witfi  this  standard,  the  term  “contractor*  refers  to  that  Qofvemmert  agency  and  the  term 
“subcontractor*  refers  to  any  contractor  of  that  Govemmerrt  agency. 

«  O  r*  standard.  This  standard  contains  a  set  of  requirements  designed  to  be  tailored  for 

iih^SS^nSdie^nScting  agency.  The  tailoring  process  Intended  for  this  standard  the 
deledon  of  noo-appHcaWe  requirements. 


■  '-v»  rvs'nji  ryr 


\rwwmwww 


0OO-STD-2ie7A  (DRAFT) 

2.  referenced  documents 


2.1  Q<y<«niniwit  docum^nta, 

2.1.1  fin4ieiHeartQns.  atandarda.  and  handbooks.  Unl*8»  Otherwise  specified,  the  following  specifications, 
standards,  and  handbooks  of  the  issue  listed  in  that  issue  of  the  Department  of  Defense  Index  of 
Specifications  and  Standards  (DOOISS)  specified  in  the  solicitation  form  a  part  of  this  standard  to  the 
extent  spedfled  herein. 

MIUTARY  STANDARDS 


DO{>-STD>2l68  •  Defense  System  Software  Quality  Program 


DOO-STD«t80 

MIL-STD-481 

MIL-STD-483 

MIL-STD-t90 

MIL>STD-499 

MIL-STD-1521 


Configuration  Control  •  Engineering  Changes,  Deviations,  and  Waivers 

Configuration  Control  *  Engineering  Changes,  Deviations,  and  Wahrers  (Short  Form) 

Configuration  Management  Practices  for  Systems,  Equipment.  Munitions,  and 
Computer  Software 

Specification  Practices 

Engineering  Management 

Technical  Reviews  and  Audits  for  Systems,  Equipments,  and  Computer  Software 


2-1.2  Other  Government  documents,  drawings,  and  Publications.  None. 

(Copies  of  specifications,  standards,  handbooks,  drawings,  and  publications  required  by  contractors  in 
connection  with  specific  acquisition  functions  should  be  obtained  from  the  contracting  agency  or  as 
directed  by  the  contracting  officer.) 

2.2  Other  Publications.  None. 

2.3  Order  of  precedence.  In  the  event  of  a  conflict  between  the  text  of  this  standard  and  the 
references  cited  herein,  the  text  of  this  standard  shall  take  precedence. 
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3.  OEF?NmON8 


3.1  ""iftii  Sm  ooo-sTo-taa 

3J  Oatwmination  by  ttw  Qov«mm«nt  that  spadfleatfon  contant  ia  accaptabla. 

3J  Saa  000-STD-(80. 

3.4  A  statamant  of  tha  charactafistlea  ol  tha  basic  alamants  of  information 
upon  by  hardwara  in  rasponding  to  computar  instrucdorw.  Thaaa  charactaristica  may  induda, 

but  ara  not  llmllad  to,  typa,  ranga.  stiuetura.  and  vaiua. 

3.5  fintnouff  hardwara.  Oavicas  capabla  of  aeeapting  and  storing  computar  data,  axacutlng  a  systamatic 
ssquanca  of  oparationa  on  computar  data,  or  produdng  control  outputs.  Such  davicas  can  parfonm 
substantial  intarpratation,  computation,  communication,  control,  or  other  logical  (Pnetions. 

3.5  fsoufcaa.  Tha  toUlKy  of  computar  hardwara,  soflwrara.  parsonnai,  documantation, 

suppllaa,  and  sarvicas  appiiad  to  a  givan  affort 

3.7  rjttnntiimr  aoSyw  (or  aoftwara).  A  combinadon  of  assodatad  computer  insbudlons  and  computar 
data  definitions  rsquirad  to  anada  tha  computar  hardwara  to  parform  computaSoiMi  or  control  functions. 

3.5  qnmfHitmr  Software  Component  (CSC1.  A  distinct  part  of  a  computer  sollwars  configuration  item 
(CSCI).  CSCs  may  be  further  decomposed  into  other  CSCs  and  Computar  Software  Units  (CSUs). 

3.5  Camotrtaf  Soltwafe  Conflouratlon  item  (CSCH.  A  configuration  Korn  for  computer  software. 

3.10  ComtuAmr  software  doeumantstlon.  Technical  data  or  Information,  (ndudfng  computar  listings  and 
printouts,  which  documents  tha  raquiramants.  design,  or  details  of  computer  software,  asplains  tha 
capabilWas  and  limitations  of  the  software,  or  provides  operating  instructions  for  using  or  supporting 
computer  software  during  tha  software's  oparationai  Ufa. 

3.11  Computer  Software  Unit  fCSU).  An  aiamant  specified  in  tha  design  of  a  Computer  Software 
Component  (CSC)  that  is  separately  testabia. 

3.12  Configuration  Identification.  See  DOO^TD-480. 

3.13  Configuration  Item.  Sea  DOD-STD480. 

3.14  Contracting  aoencv.  As  used  in  this  Standard,  contracting  agency  refers  to  tha  'contracting  office" 
as  defined  in  Federal  Acquisition  Regulation  Subpart  2.1,  or  its  designated  representative. 

3.15  Developmental  Configuration.  The  contractor's  software  and  associated  technical  documentation 
that  defines  the  evolving  configuration  of  a  CSCI  during  development.  It  is  under  the  development 
contractor's  configuration  control  and  describes  the  software  design  and  Implementation.  The 
Developmental  Configuration  for  a  CSCI  consists  of  a  Software  Design  Document  and  source  code 
listings.  Any  Kern  in  the  Developmental  Configuration  may  be  stored  on  electronic  media. 

3.18  gvaluatlon.  See  DOO^TD'2168. 

3.17  Fimiware.  The  combination  of  a  hardware  device  and  computer  instnictions  or  Mmputer  data  that 
reside  as  read-only  software  on  the  hardware  device.  The  software  cannot  be  readily  modified  under 
program  control. 
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T^n«  ffOP.  A  ^  .g-Ky  to  d—mlo. 

th«r  •  oonflQurttton  ttvm  comp«««  with  th*  iHoeiMa  r«quirwn«ntt  for  vm  itom. 
e.,.#^iHnai--nflni  Sm  OOO-STD-ISO. 

ConWfltiffton  Itom  (HWCH.  A  cortlgorrtloo  llom  tor  h«idv«f». 


3^1  ind«c«ndont  VftflcaHon  and  v*wdtton  flViVt.  Vorlllcotloo  and  villdaBon  pwfbrmod  by  a  contrac- 
tor  or  Oovammant  agancy  that  ia  not  rasponaibla  for  davaJoping  tha  product  or  paiforming  tha  activity 
baing  avaiuatad.  IV&V  is  an  activity  that  is  conducted  saparataly  from  tha  softwara  davaiopmant 
activitias  govamad  by  this  standard. 

3^  Moivdavaiopmantai  soffwara  <NOS).  Oaiivarabla  aoAwara  that  is  not  davalopad  undar  tha  contract 
but  is  providad  by  tha  contractor,  tha  Oovammant,  or  a  third  party.  NDS  may  ba  rafarrad  to  as 
rausabia  softwara,  Oovammant  fumishad  sofNvara,  or  commarcially  avaiiabia  softwara,  dapanding  on  its 
sourca. 


3.23  Product  Bsaaiina.  Saa  OOO^TD-480. 

3.24  Raiaaaa.  A  configuration  managamant  action  wharaby  a  particular  vatsion  of  sothvara  Is  made  V. 
avaiiabia  ter  a  apadfle  purpoaa  (a.g.,  raiaasad  ter  tast). 

3.25  •nfiwara.  Sofiwara  davalopad  in  raaponaa  to  tha  raquiramanls  ter  ona  application  that  .’;Z 
can  ba  uaad,  in  whoia  or  in  part  to  aatisly  tha  raquiramanls  of  ruMthar  appBcaHon. 

3  Sofrwara  davaloomant  fils  (SDR.  A  rapository  ter  a  coMaction  of  matartat  partinant  to  tha  > 
davaiopmant  or  support  of  softwara.  Contants  typi^ly  induda  (aithar  diractiy  or  by  rateranca)  design  ^ 
conaidarationa  and  constraints,  design  documaritation  and  data,  schedule  arid  sta^  information,  tast 
raquiramants,  test  eaaaa,  tast  procaduraa,  and  tast  rssuHs.  ^ 

3.27  Software  davaiopmant  library  (SDU.  A  controllad  collaction  of  software,  documentation,  and  -■ 

assodatad  tools  and  procedures  used  to  fadlitata  the  ordaily  davaiopmant  arul  subsequent  support  of 
softwara.  Tha  SDL  indudas  tha  Oavaiopmantai  Configuration  as  part  of  its  contants.  A  software  ^ 
davaiopmant  library  provides  storage  of  and  controllad  access  to  softwara  and  documentation  in  'S 
human-raadabia  term,  machina-raadabla  term,  or  both.  Tha  library  may  also  contain  management  data 
partinant  to  tha  software  development  project  ^ 

3.28  Software  anaineartno  environment  Tha  sat  of  automated  tools,  firmware  devices,  and  hardware 
necessary  to  perform  tha  softwara  engineering  effort  Tha  automated  tools  may  induda  but  are  not  , 
limited  to  compilers,  assemblers,  linkers,  toadars,  operating  system,  debuggers,  simulators,  emulators,  test 
tools,  documentation  tools,  and  data  base  managamant  systam(s). 

3.29  Software  support  Tha  sum  of  all  adivities  that  taka  place  to  ensure  that  implemented  and  fielded  ^ 
softwara  continues  to  fully  support  tha  operational  mission  of  tha  software. 

3  JO  Software  test  environment.  A  set  of  automatad  tools,  firmware  devices,  and  hardware  necessary  to 
test  softwara.  Tha  automated  tools  may  induda  but  are  not  limited  to  test  tools  such  as  simulation 
software,  coda  analyzers,  etc.  and  may  also  induda  those  tools  used  in  the  softwara  engineering  en- 
vironmant 

.V 

3.31  System  Specification.  A  system  level  requirements  specification.  A  system  specification  may  be  a  ‘-j 
System/Sagment  Specification  (SSS),  Prime  Item  Development  Specification  (PIDS),  or  Critical  Item 
Development  Specification  (CIDS).  i 
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3Ja  TTi*  procw  of  ovoluoting  sdtm  to  dotwmlno  compllonco  with  sp^MM  roquiramm 

ajsa  ViinriHim  “O'*  procw*  of  •vtluttlng  th*  products  of  a  givon  softwars  davalopmant  activity  to 
^^^^^^"^orrartnaat  and  oooaiatancy  with  nspact  to  tha  products  and  standarda  provided  as  input  to 


that  activity. 


<1^  Vafsion  An  idantiflad  and  documantad  body  of  softwara.  Modifications  to  a  version  of  software 
(laauSngT  a  new  version)  require  configuration  management  actions  by  eithar  the  contractor,  the 
contracting  agency,  or  both. 
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4.  GENERAL  REQUIREMEffTS 


Th«  contrtctor  shall  parform  softwara  development  management 


in  compliance  wtti  the  following  requirements. 

4.1 1  dm/alnnmunt  process.  The  contractor  shall  implement  a  process  for  managing  the 

devek»^^  ^  deliverable  soft^.  The  contractor's  software  development  process  for  each  CSCI 
shall  be  compatible  wrth  the  contract  schedule  tor  formal  reviews  and  audits.  The  software  development 
process  shall  indude  the  following  major  activities,  which  may  overlap  and  may  be  applied  iteratively  or 
lecuisiveiy: 

a.  System  Requirements  Analysis/Design 

b.  Software  Requirements  Analysis 

e.  Preliminary  Design 

d.  Detailed  Design 

e.  Coding  and  CSU  Testing 

f.  CSC  integration  and  Testing 

g.  CSCI  Testing. 

h.  System  Integration  and  Testing. 

4.1.2  Pomisi  fwiewa/sudita.  During  the  software  development  process,  the  coittractor  shaO  conduct  or 
support  formal  reviews  and  audits  as  required  by  the  contracL  Guidance  on  formal  reviews  and  audita  is 
provided  In  MIL*STD*1521.  The  relationship  of  the  formal  reviews  and  audHs  to  software  and  hardware 
development  is  shown  in  Rgure  1.  Figure  2  illustrates  the  occurrence  of  formal  reviews  arfo  audits  for 
software  and  shows  the  relationship  of  deliverable  products  to  baselines  and  the  Developmental 
Conflguration. 

4/ j  SofN»sre  develooment  olannina.  The  contractor  shall  develop  plans  for  conducting  the  acdvilles 
required  by  this  standard.  These  plans  shall  be  documented  in  a  Software  Development  Plan  (SDP). 
Following  contracting  agency  approval  of  the  SDP,  the  contractor  shall  conduct  the  activities  required  by 
this  standard  in  accordance  with  the  SDP.  Wifo  the  exception  of  scheduling  information,  updates  to  the 
SDP  shall  be  subject  to  contracting  agency  approval. 

4.1.4  Biak  management.  The  contractor  shall  document  and  implement  procedures  for  risk  management. 
The  contractor  shall  identify,  analyze,  prioritize,  and  monitor  the  areas  of  the  software  development 
project  that  involve  potential  technical,  cost,  or  schedule  risks. 

4.1 .5  Security.  The  contractor  shall  comply  with  the  security  requirements  specified  in  the  contract. 

4.1.5  Subcontractor  management.  The  contractor  shall  pass  down  to  the  subcontractor(s)  all  contractual 
requirements  necessary  to  ensure  that  all  software  and  associated  documentation  delivered  to  the 
contracting  agertcy  are  developed  in  accordance  with  the  prime  contract  requirements.  The  contractor 
shall  provide  to  the  subcontractor(s)  the  baselined  requirements  for  the  software  to  be  developed  by  the 
subcontractor(s). 

4.1.7  Interface  with  software  IV&V  aoentfsl.  The  contractor  shall  interface  with  the  software 
Independent  Verification  and  Validation  (IV&V)  agent(s)  as  specified  in  the  contract. 

4.1.8  Cost/schedule  reporting.  The  contractor  shall  prepare  and  maintain  cost/schedule  reports  at  the 
CSCI  level.  The  cost  reports  shall  indicate  budgeted  versus  actual  expenditures  and  shall  conform  to  the 
Work  Breakdown  Structure  (WBS)  applicable  to  the  development  effort.  These  reports  shall  also  indicate 
to  the  contracting  agency  planned,  actual,  and  predicted  progress. 


I. 
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4.1.9  Softww  d«valQpm«nt  librafv.  Th«  contractor  shall  aatabiish  a  soltwara  davalopmant  library  (SOL) 
Tha  contractor  shall  document  and  impl^ant  procaduraa  Ibr  controlHng  software  and  aasodatad 
documentation  residing  within  the  SOL  The  contractor  shall  maintain  the  SOL  for  the  duration  of  the 
contract 

4.1.10  Corrective  acBon  process.  The  contractor  shall  document  and  implement  a  corrective  action 
procesa  for  handling  all  problems  detected  in  the  products  under  configuration  control  and  in  the 
software  development  activities  required  by  the  contract  The  corrective  action  process  shall  comply 
with  the  following  requirements: 

a.  The  process  shall  be  closed-loop,  ensuring  that  ail  detected  problems  are  promptly  reported  and 
entered  Into  the  corrective  action  process,  action  Is  Initiated  on  them,  resolution  is  achieved, 
status  is  tracked  and  reported,  and  records  of  the  problems  are  maintained  for  the  life  of  the 
contract 

b.  Inputs  to  the  corrective  action  process  shall  consist  of  proWem/change  reports  and  other 
discrepancy  reports. 

c.  Each  problem  shall  be  classified  by  category  and  by  priority.  The  categories  and  priorities 
identified  in  Appendix  C  shall  be  included  in  the  category  and  priorily  dassiflcatlons. 

d.  Analysts  shall  be  performed  to  detect  trends  in  the  problems  reported. 

e.  Corrective  actions  shall  be  evaluated  to:  (1)  verify  that  problems  have  been  resolved,  adverse 
trends  have  been  reversed,  and  changes  have  been  correctly  implemented  In  the  appropriate 

processes  and  products,  and  (2)  to  determine  whether  additionai  problems  have  been  introduced, 

% 

R*Tibl9fn/ch9n00.  report,  The  contractor  shall  prepare  a  problem/change  report  to  describe  each 
problem  detected  in  software  or  documentation  that  haa  been  placed  under  configuration  control.  The 
probiem/change  report  shall  describe  the  corrective  action  needed  and  the  actions  taken  to  resolve  the 
problem.  These  reports  shall  serve  as  input  to  the  corrective  action  process. 

Software  engineering,  the  contractor  shall  perform  software  engineering  in  compliance  with  the 
following  requirements. 

*■2'^  Software  development  methods.  The  contractor  shall  use  systematic  and  well  documented  software 
development  methods  to  perform  requirements  analysis,  design,  coding,  integration,  and  testing  of  the 
deliverable  software.  The  contractor  shall  implement  software  development  methods  that  support  the 
formal  reviews  and  audits  required  by  the  contract 

4.2.2  Software  enoineerino  environment  The  contractor  shall  establish  a  software  engineering 
environment  to  perform  the  software  engineering  effort.  The  software  engineering  environment  shall 
comply  with  the  security  requirements  of  the  contract  The  contractor  shall  document  and  implement 
plans  for  the  installation,  configuration  control,  and  maintenance  of  each  item  of  the  environment. 

4-2-3  Safety  analysis.  The  contractor  shall  perform  the  analysis  necessary  to  ensure  that  the  software 
requirements,  design,  and  operating  procedures  minimize  the  potential  for  hazardous  conditions  during  the 
operational  mission.  Any  potentially  hazardous  conditions  or  operating  procedures  shall  be  dearly 
identified  and  documented. 
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4J4  aaltmM.  Th«  coHrictor  »h<t  coiwldf  IncocponUnfl  non-d«««topnMntai 

Mo  th«  (MN«rabi«  soltMT*.  Th«  oontr^aor  shtB  document  plan*  for  uslo®  N08.  ND8 
may  ba  liiujmonrtd  by  ttw  contractor  wMwtA  contracting  agancy  approval  only  tf  tha  NOS  ia  IbOy 
documanlad  hi  aeoowlMwa  wHh  tha  raquiramanta  of  IMa  standard.  Tha  sofNrara  davafopmant  flias  tor 
NOS  naad  not  oenWn  ttw  daalgn  conaldaraliona.  constrainta.  or  data.  Incoiporation  of  NOS  shaH  comply 
with  tha  data  rtgMi  raquiramartts  in  tha  contract 

4^5  ftonintitf  aoltwfa  Qfoanizitlon.  Tha  contractor  shaH  daoompoaa  and  partition  aach  CSCI  Into 
Computar  Softwara  Componants  (CSCs)  and  Computar  Sotowara  UnKa  (CSUa)  in  aocordanca  with  tha 
davatopmant  mathod(a)  documantad  in  tha  Softwara  Oavatopmant  Plan  (SOP).  Tha  contractor  shall  ansura 
that  tha  raquiramanta  tor  tha  CSC!  ara  complataiy  ailocatad  and  furthar  rallnad  to  todlitata  tha  dasign 
and  taat  of  aach  CSC  and  CSU.  Rgura  3  praaanta  an  Utustratlon  of  a  syatam  braakdown  and  CSCI 
daoompoaition. 

4.2.6  Traeaabilltv  of  raouifamanta  to  daalon.  Tha  contractor  shall  document  tha  tracaabilNy  of  tha 
raquiramanta  ailocatad  from  tha  system  spacHIcatlon  to  aach  CSCI,  tta  Computar  Softwara  Componants 
(CSCs)  and  Computar  Software  Units  (CSUa),  and  ftom  tha  CSU  lava!  to  tha  Softwara  Raquiramants 
Spadllcatlona  (SRSs)  and  intarfaca  Raquiromarfts  SpacWcatlon  (IRS). 

4.2.7  High  oidaf  lanausoa.  Tha  contractor  shall  use  the  High  Order  Languaga(s)  (HOLs)  apadflad  in  tha 
contract  to  coda  tha  daiivarabla  software,  if  no  HOL  la  raquirsd  by  tha  contract  tha  corrtractor  shall 
obtain  contracting  agancy  approval  to  use  a  particular  language. 

4.2.8  Daalon  and  eedlno  standards.  Tha  corftractor  shaH  document  and  Implamant  dasign  and  coding 
standards  to  ba  used  in  tha  davalopman:  of  daiivarabla  softwara.  Softwara  coding  staiMlarda  shaH  comply 
with  tha  raquiramants  spadflad  in  Appendix  B. 

4.Z9  Softwara  davalooment  files.  Tha  contractor  shaH  document  tha  davalopmartt  of  each  Computer 
Software  Unit  (CSU).  Computer  Softwara  Component  (CSC),  and  CSCI  in  softwara  davalopmant  files 
(SOFs).  Tha  contractor  shall  establish  a  separata  SDF  tor  aach  CSU  or  a  logically  ralatad  group  of 
CSUa;  aach  CSC  or  a  logically  ralatad  group  of  CSCs;  and  aach  CSCI.  Tha  contractor  shall  document 
and  implamant  procedures  for  establishing  and  maintaining  SOFs.  Tha  contractor  shall  maintain  the  SDFs 
for  the  duration  of  tha  contract.  Tha  SOFs  shall  ba  made  available  for  contracting  agency  raviaw  upon 
request  SOFs  may  bo  generated,  maintained,  and  controlled  by  automated  means.  To  reduce  duplication, 
SOFs  should  not  contain  information  provided  in  other  documents  or  SOFs.  The  set  of  SOFs  shall 
include  (directly  or  by  reference)  the  following  information: 

a.  Design  considerations  and  constraints 

b.  Design  documentation  and  data 

c.  Schedule  and  status  information 

d.  Test  requirements  and  responsibilities 

e.  Test  cases,  procedures,  and  results. 

4.Z10  rasouree  and  reserve  caoacitv.  The  contractor  shall  analyze  the  processing  resource 

and  raearva  raquiramants,  such  as  timing,  memory  utilization,  I/O  channel  utilization,  idertofisd  in  the 
contract  and  shall  allocate  these  resources  among  the  CSCIs.  The  allocation  of  these  resources  to  a 
CSCI  shall  ba  documented  in  the  Software  Requirements  Specifications  (SRSs)  tor  that  CSCI.  The 
contractor  shall  monitor  the  utilization  of  processing  resources  for  tha  duration  of  the  contract  and 
shall  reallocate  the  resources  as  necessary  to  satisfy  the  reserve  requirements.  Measured  resource 
utilization  at  the  time  of  delivery  shall  be  documented  in  the  Software  Product  Specification  (SPS)  tor 
each  CSCI. 
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FIGURE  3 
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1 1  rnim^  7h«  cortrictof  sha*  conduct  FOT  of  ••ii  CSa  on  th#  t»g«t  compute 

Mtom  MMuivai^  ty^  cpprov^t  by  tho  contracting  cgwiey.  TTra  contractor's  FQT  actMUM 
Mud*  Mwaina  tho  sothrara  M  tho  Iknlla  of  Mo  apocMod  raqutoornonto.  Tho  contractor  may 
oonduol,  00  p«t  of  Iho  FQT  octMty,  tooting  of  CSCIo  mtogralod  with  othor  CSCis  or  HWCJo  that 
comprioo  tho  syolam. 


4A1  QuaiWraHon  Taat  Planning.  Tho  contractor  shaH  dovolop  plana  for  conducting  tha  formal 

quaWIcation  tooting  (FQT)  activitiaa  raquirod  by  thia  standard.  Tbooo  plana  shall  bo  documontod  in  tho 
Softwara  Toot  Plan  (STP).  Following  oontracting  agoney  approval  of  tho  STP.  tho  contractor  shall 
conduct  tho  FOT  acdvitios  in  aceordanco  with  tho  STP.  With  tho  oxcoption  of  schodullng  information, 
updalos  to  tho  STP  shall  bo  subfoct  to  oontracting  agoney  spprovoL  Tho  contractor  shall  idontity  In  tho 
Sollwara  Toot  Pt«i  (STP)  tho  tosts  that  involvo  straoaing  tho  aoAwara  and  thoao  that  Involvo  int^rating 
CSCIa  with  othor  oor^guration  ftams. 


A  1^9  Se/twmrm  toat  anvlronmanL  Tho  contractor  shall  aatobllah  a  soitwaro  tost  anvironmont  to  porform 
tho  FQT  oltart  Tho  softwaro  tost  anvironmont  shaN  comply  with  Iho  aocurity  roquirsmonts  of  tho 
contract  Tho  contractor  shall  documont  and  implofflont  plam  tar  tho  installation,  tost  configuration 
cerOrol,  and  maintonanco  of  oach  Mom  of  tho  onvirorunont  Fotowing  InstaRation,  ooch  Mam  of  tho 
anvironmont  shall  bo  tostod  to  domonstrato  that  tho  ftam  portarma  Ns  intondod  tanetion. 


Indooondoneo  In  FQT  activitioa.  Tho  organizations,  tanctiona,  and  parsons  rasponsiblo  tar  fullBling 
tho  FQT  roquiromonta  of  this  standard  shaN  havo  tho  rooourcoo,  raaponaibilMy,  authority,  and 
organizational  fToodom  to  pormit  obiocttvo  toadng  and  to  causa  tho  inttlfllion  aiwi  vorMeatfon  of 
oorroctivo  actioa  Tho  parsons  conducting  FQT  acdvitioa  shaN  ndl  bo  tho  ponona  who  dovoiopod  tho 
softwaro  or  ara  rasponsiblo  for  tho  softwara.  This  dooo  not  praduda  mom  bars  of  tha  sotaMara 
onginooring  taam  bom  participating  in  FQT  activitios. 


4J.4  Tfseoabilitv  of  roQuifom<rP*1  Iff  t***  Tho  contractor  shaU  documont  tho  tracoabHlty  of  tho 

roquiramonts  in  tho  Softwara  RtquiremBnia  Spacifications  (SRSs)  and  intartaco  Raquiramonts  Spocification 
(IRS)  that  aro  satisfiod  or  partially  satisflad  by  aach  tost  caso  idontiflod  in  tho  Softwaro  Tost 
Ooscription  (STD).  Tho  contractor  shall  documont  this  tracoabiiity  in  tho  STD  for  oach  CSCI. 


4.4  Software  product  a-valuations.  Tho  contractor  shall  cortduct  ovaiuations  of  dolivorablo  softwara  and 
documontation  as  spociHed  in  soctlon  5  of  this  standard  and  in  compliance  with  tho  following 
roquiromants. 


4.4.1  Indeoendenea  in  product  evaluation  activitioa.  Tho  organizations,  functions,  and  parsons 
rasponsiblo  for  fulfilling  tho  evaluation  roquiramonts  of  this  standard  shall  havo  tho  resources, 
responsibility,  authority,  and  organizational  fiWlom  to  pormit  objoctivo  evaluation  and  to  causa  the 
initiation  and  verification  of  corroctivo  action.  The  persons  conducting  tho  evaluation  of  a  product  shall 
not  bo  tho  porsorts  who  dovaloped  tho  product  or  aro  rasponsiblo  for  tho  product.  This  does  not 
pradudo  mombors  of  the  dovalopmont  team  from  participating  in  thosa  evaluations.  Responsibility  fcr 
tho  fuifillmont  of  tho  software  product  evaluation  requirements  shall  bo  assigned  and  specified  in  the 
Softwaro  Oovolopmont  Plan  (SOP). 


4.4.2  Rnai  evalueWons.  Prior  to  submitting  aach  dolivorablo  item  to  tho  contracting  agency,  tha 
contractor  shall  internally  coordinate  the  item  with  appropriate  organizations  for  a  final  evaluation.  The 
objoctivo  of  each  final  evaluation  shall  bo  to  ensure  that  tho  dolivorablo  item  is  accoptablo  In  terms  of 
its  ability  to  satisfy  the  need  for  which  it  was  developed.  v'' 


4.4.3  Software  evaluation  records.  Tho  contractor  shall  prepare  and  mairrtain  records  of  oach  software 
product  ovaluation  performed.  When  problems  have  been  identified  a  problem/change  report  shall  be 
inltiatad  and  shall  servo  as  input  to  tho  corrective  action  process.  The  evaluation  records  shall  be 
available  for  contracting  agency  review  and  shall  bo  maintained  for  the  life  of  the  contract.  * 
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.  ^  ^  Th«  contnctof  sh««  •v«lu«t*  th«  products  idsntiflsd  »n  s«:tion  S  agsinst  th« 

,  in  Flguras  4  through  10.  Osfauit  dsfinitions  for  th*  crftsris  are  spaciflad  lr» 

loMndte  0  Tha  may  propoaa  additional  critarta  and  aHamata  daflnitiona  for  any  of  tha 

Courts  agarlflail  in  Appandix  0.  Additional  crtaria  and  altamata  daflnitiona  ara  aubjact  to  contracting 
agoncy  approval. 

18  cQnflnunrtlon  manapamant  Th*  contractor  shall  parform  softwara  configuration 

managamant  In  complianca  with  tha  following  raquiramants. 

45  <  CanHauntion  iH^ntfflgation.  Tha  contractor  Shall  documant  and  Implamant  plans  for  parforming 
configuration  idantiflcatlon.  Configuration  idantHIcation  shall  ba  conductad  in  accordanca  with  tha 
Idantiflcatlon  schama  spaciflad  in  tha  contract  Configuration  idantHIcation  parfbnnad  by  tha  contractor 
shall  accomplish  tha  following: 

a.  IdantHy  tha  documantation  that  astabiishas  tha  Functional,  Allocatad,  and  Product  Basalinas,  and 
tha  Oavaiopmantal  Configuration. 

b.  Idantify  tha  documantation  and  tha  computar  softwara  madia  containing  coda,  documantation,  or 
both  that  ara  piacad  undar  configuration  control. 

c.  IdantHy  aach  CSCI  and  its  corresponding  Computar  Softwara  Components  (CSCs)  and  Computar 
Software  Units  (CSUs). 

d.  IdantHy  tha  version,  ralaasa,  change  status,  and  any  other  idontHIcation  details  of  aach 
daUvarabla  Ham. 

a.  Identity  tha  version  of  aach  CSCI.  CSC,  and  CSU  to  vrhich  tha  corresponding  software 
documantation  applies. 

f.  IdantHy  tha  specific  version  of  softwara  contained  on  a  daUvarabla  medium.  Including  all 
changes  incorporated  since  its  previous  ralaasa. 

4.5.2  Conflauration  control.  The  contractor  shall  document  and  implement  plans  for  performing 
configuration  control.  Configuration  control  performed  by  tha  contractor  shall  accomplish  tha  following: 

a.  Establish  a  Developmental  Configuration  for  aach  CSCI. 

b.  Maintain  current  copies  of  the  deiiverabla  documentation  and  code. 

c.  Provide  the  contracting  agency  access  to  documentation  and  code  under  configuration  control. 

d.  Control  tha  preparation  and  dissemination  of  changes  to  the  master  copies  of  deliverable 
software  and  documentation  that  have  been  piacad  under  configuration  control  so  that  they 
reflect  only  approved  changes. 

4.5.3  Configuration  status  accounting.  Tha  contractor  shall  document  and  implement  plans  for 
parforming  configuration  status  accounting.  Tha  contractor  shall  generate  management  records  and 
status  raports  on  all  products  comprising  tha  Oavaiopmantal  Configuration  and  the  Allocated  and  Product 
Basalinas.  Tha  status  reports  shall: 

a.  Provide  traceability  of  changes  to  controlled  products. 

b.  Serve  as  a  basis  for  communicating  the  status  of  configuration  identification  and  associated 

software. 


17 


n.-*  'VT'  V* 


I  I  I  ■  II  lumiii......  I, _ _ 

"  I  HI  lin— Mpri-Ii _ 

u-'WJW*li^irU»y»U,wrv-TimrwTwwnDnsrri 


000>6nrO-2ie7A  (DRAFT) 


c.  S«(v«  M  •  vahici*  for  onsuring  that  daifvartd  documants  dMcrfba  «nd  raprasant  tha  aaaodalad  ^ 

Mflwar*. 


4^  ytwoa.  handltno.  and  dalh/mrv  ol  Dfota^  madia.  Tha  contractor  ahall  documant  and  impiamant 
maihoda  and  prooaduraa  for  tha  storaga,  handling,  and  daitvary  of  aoltwara  and  documantation.  Tha  ^ 
contractor  ahal  maintain  maatar  copias  of  tha  dalivarad  softwara  and  documantation.  ^ 

A5J  EnoinaafinQ  Chanoa  Proposals.  Tha  contractor  ahall  prapara  Enginaaring  Changa  Propoaala  (ECPs) 
in  aoeordanca  with  MIL*8TTM83,  000>STD-<80,  and  MIL-STD<461.  Tha  contractor  ahail  prapara 
Spadflcatlon  Changa  Nodcaa  (SCNs)  in  accordanca  with  MIL-ETD-483  and  MIL-STD-490. 

4.8  TmrtteninQ  to  softwara  support.  Tha  contractor  ahaM  prowida  tranaMon  auppoit  in  compttanca  '! 
with  tha  following  raquiramants. 


4A1  RtgWMittt  inri  miifrtlinittt  cadt-  Tha  contractor  ahaN  provida  to  tha  contracting  agancy 
daHvarabia  coda  that  can  ba  reganaratad  and  maintainad  using  oommardalfy  availabia,  Qovammant- 
ownad,  or  corrtractualiy  daliveraWa  support  softwara  and  hardwara  that  has  baan  idaiTtiflad  by  tha 
contracting  agancy. 


A6.2  TflfWftten  Olinr'^^-  Tha  contractor  shall  prapara  plana  for  transitioning  tha  dailvarabla  softwara 
from  dawaiopmant  to  support  Thasa  plana  shall  ba  documantad  In  tha  Computar  Raaourca  Intagratad 
Support  Documant  (CRISO). 


4.SJ  Softarara  tranaitiQn  and  continuing  suocxart  Tha  contractor  shaH  parform  Instaflabon  and  chackout 
of  tha  dailvarabla  software  in  tha  support  anvironmant  daaignatad  by  tha  contracting  agancy.  Tha  y 
contractor  shall  provida  training  and  continuing  assistartca  to  tha  corttractino  agancy’s  support  activity  '> 
as  spaeWad  In  tha  contract 


4-8.4  Saftjfara  support  and_  poarst/onsl  documantation.  Tha  contractor  shall  davalop  and  dalivar  tha 
following  softwara  support  and  oparationai  documantation  as  raquirad  by  tha  Contract  Dsta  Raquiramants 
Ust  (CORg: 


i 


a.  Computar  Resources  Integrated  Support  Document  (CRISO) 

b.  Computer  System  Operator’s  Manual  (CSOM) 

c.  Softwara  User's  Manual  (SUM) 

d.  Software  Programmer's  Manual  (SPM) 
a.  Frmwara  Support  Manual  (PSM). 
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S.  OETAIL£D  REOUIREMEhfTS 

5j  ra^ilfmanti  analvsla/d—ion.  Th*  contractor  shail  parform  th*  following  system  rsquirsments 

snalysia/dasign  activiUM. 

5.1.1  Software  dayeloomant  rnsnaaemant. 

5.1. 1.1  The  contractor  shall  support  the  System  Requirements  Review  (SRR)  as  specified  in  the 
contract 

5.1. 1.2  The  contractor  shall  support  the  System  Design  Review  (SDR)  as  specHfed  In  the  contract 

5.1.2  Software  engineering. 

5.1 .2.1  The  contractor  shall  analyse  the  preliminary  system  specification  and  shail  determine  whether 
the  software  requirements  are  consistent  and  complete. 

5.1.22  The  contractor  shall  conduct  analysl*  to  determine  the  beet  allocation  of  system  requirements 
between  hardware,  software,  and  personnel  order  to  partition  the  system  Into  HWCIs,  C^ls,  ai>d 
msnuSl  operations.  The  contractor  shall  docurr.y- '  the  allocation  in  a  System/Segment  Design  Documerrt 
(SSOO). 

t 

5.122  The  contractor  shall  define  a  preliminary  set  of  engineering  requirements  for  esch  CSCL  The 
contractor  shall  document  these  requirements  in  the  preliminary  Softwm  Requirements  Spedfication 
(SRS)  for  each  C3CI. 

5.1.24  The  contractor  shall  define  a  preliminary  set  of  interface  requirements  for  each  external 
interface  of  each  CSCI.  The  contractor  shall  document  these  requirements  in  a  preliminary  Interface 
Requirements  Specification  (IRS). 

5.12  Formal  qualification  testing.  The  contractor  shail  define  a  preliminary  sot  of  qualification 
requirements  for  each  CSCI.  The  contractor  shall  document  these  requirements  in  the  preliminary 
Software  Requirements  Specification  (SRS)  for  each  CSCI.  These  requirements  shall  be  consistent  with 
the  qualification  requirements  defined  in  the  system  specification. 

5.1.4  Software  product  evaluations.  The  contractor  shall  perform  evaluations  of  the  following  products, 
using  the  evaluation  criteria  specified  in  Figure  4: 

a.  The  Software  Development  Plan  (SOP) 

b.  The  System/Segment  Design  Document  (SSOO) 

c.  The  preliminary  Software  Requirements  Specification  (SRS)  for  each  CSCI 

d.  The  preliminary  Interface  Requirements  Specification  (IRS). 

5.1.5  Conflouratlon  management.  The  contractor  shall  place  the  following  documents  under  configuration 
control  prior  to  delivery  to  the  contracting  agency; 

a.  The  Software  Development  Plan  (SDP) 

b.  The  System/Segment  Design  Document  (SSDD) 

c.  The  preliminary  Software  Requirements  Specification  (SRS)  for  each  CSCI. 

d.  The  preliminary  Interface  Requirements  Specification  (IRS). 
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Th«  contractor  shall  parfbnn  tha  following  softwara  raquiramants 


5.2  foiiirnmanta  anaivaia.  in»  - - - -  - - 

•nalyaia  actlvMlaa* 

,  manacamaot.  Tha  contractor  shall  conduct  ona  or  mora  Softwara 

q£L^H^S^naijla^^(^R)  acc^^an^  with  MIL-Srn>1521.  FollowInQ  succassful  complation  of  an 
SSRaifo^an  authanticated  by  tha  contracting  agancy,  tha  Softwara  Raquiramants  Spaciflcations  (SRSs) 
and  associatad  Intarfaca  Raquiramants  SpacfflcaMon  (IRS)  will  asUbiish  tha  Allocatad  Basalina  for  tha 
CSCIs. 


K991  Tha  contractor  shall  dafina  a  comptata  sat  of  anginaaiing  raquiramants  for  each  CSCI.  Tha 
contractor  shall  documant  thasa  raquiramants  in  tha  Softwara  Raquiramants  Spacification  (SRS)  for  each 
CSCL 

5200  Tha  contractor  shall  dafina  a  compiata  sat  of  Intarfaca  raquiramants  for  each  external  interface 
of  each  CSCI.  Tha  contractor  shall  documant  thasa  raquiramants  in  an  Intarfaca  Requirements 
Spadflcation  (IRS). 

5,23  Fomial  miailfieetlQn  tasting.  Tha  contractor  shaH  dafina  a  compiata  sat  of  qualification 
raquiramants  for  aach  CSCI.  Tha  contractor  shaH  document  these  raquiramants  in  tha  Softwara 
Raquirarrtants  Spacification  (SRS)  for  aach  CSCI. 

52^  Software  omHuet  evaluations.  Tha  contractor  shaH  perform  evaluations  of  tha  products  Idantifiad 
below,  using  tha  evaluation  criteria  spadfiad  in  Rgura  5.  Tha  contractor  shall  present  a  summary  of 
tha  evaluation  results  at  the  Softwara  Spacification  Raviaw(s). 

a.  The  Softwara  Raquiramants  Spacification  (SRS)  for  aach  CSCI 

b.  Tha  Intarfaca  Requirements  Spacification  (IRS). 

5,2.5  Conflouretlon  management.  Tha  corrtractor  shall  place  tha  Softwara  Raquiramants  Specification 
(SRS)  for  each  CSCI  and  tha  associatad  intarfaca  Raquiramants  Spacification  (IRS)  under  configuration 
control  prior  to  delivery  to  the  contracting  agancy. 
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5J  pntlimi"*~  ^***°"-  Th*<»ntr«etof8htllpwfonnth«followinQpr*llmin«ryd«»lgn«ctlv« 

3J  .,  dawinnmfrt  m«n«QTn«ot  Th«  contTMtof  *hall  conduct  00#  or  mor*  Prrtimintfy  D«t<on 

Ravi«w(s)  (POR)  In  accordanco  with  MIL-STO-1521. 

SJ.2  Sflftwaw  •nolnearinQ. 

5J.2.1  Tha  contractor  shall  develop  a  preliminafy  design  for  each  CSCI.  shall  allocate  requirements 
from  the  Software  Requirements  Speciflcatlona  (SRSa)  and  associated  Interface  Requirements 
Specifications  (IRS)  to  the  CSCs  of  each  CSCI.  and  shall  establish  design  requirements  for  each  CSC. 
The  contractor  shall  document  this  information  in  the  Software  Design  Document  (SOD)  for  each  CSCI. 

SJ.2.2  The  contractor  shall  develop  a  preliminary  design  for  the  external  interfaces  of  each  CSCI 
documented  In  the  Interface  Requirements  Specification  (IRS).  The  contraaor  shall  document  this 
information  in  a  preliminary  Interface  Design  Document  (IDD). 

?)  g  a  The  contractor  shall  document  any  additional  engineering  information  generated  during  the 
proliminaty  design  process  in  Section  6  of  the  Software  Design  Document  (SOD)  for  each  CSCI.  The 
engineering  information  shall  include  rationale,  results  of  analyses  and  tradiHOlT  studies,  and  any  other 
information  that  aids  in  understanding  the  preliminary  design. 

SJ.2.4  The  contractor  shall  establish  test  requirements  for  conducting  CSC  integration  and  testing. 
The  contractor's  CSC  integration  and  testing  shall  include  stressing  the  software  at  the  limits  of  its 
specillod  requirements.  The  contractor  shall  record  the  test  requirements  (directly  or  by  reference)  in 
the  CSC  software  development  files. 

5JJ  Formal  aualifleation  testing.  The  contractor  shall  identify  the  formal  qualification  tests  to  be 
conducted  to  comply  with  the  qualification  requirements  identified  in  the  Software  Requirements 
Specification(s)  (SRSs).  The  contractor  shall  document  this  information  for  each  CSCI  in  the  Software 
Teat  Plan  (STP). 

5J.4  Software  product  evaluations.  The  contractor  shall  perform  evaluations  of  the  products  identified 
below,  using  the  evaluation  criteria  specified  In  Rgure  6.  The  contractor  shall  present  a  summary  of 
the  evaluation  results  at  the  Preliminary  Design  Review(s). 

a.  The  Software  Design  Document  (SDO)  for  each  CSCI 

b.  The  preliminary  Interface  Design  Document  (IDD) 

c.  The  Software  Test  Plan  (STP) 

d.  The  CSC  test  requirements. 

5.3.5  Configuration  management. 

5.3.5.1  The  contractor  shall  incorporate  the  Software  Design  Document  (SDD)  for  each  CSCI  into  the 
cscrs  Developmental  Configuration  prior  to  delivery  to  the  contracting  agency. 

5.3.5.2  The  contractor  shall  place  the  Software  Test  Plan  (STP)  under  configuration  control  prior  to 
delivory  to  the  contracting  agency. 

5.3.5.3  The  contractor  shall  place  the  preliminary  Interface  Design  Document  (IDD)  under  configuration 
control  prior  to  delivery  to  the  contracting  agency. 
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M  H^an.  Th«  cortrMtof  Shan  p«iform  th«  following  dotaiiod  dotign  aciivilio*. 

5.4.1  PwiwiOBmant  Manaoomoflt  Tho  cootTMtof  thall  conduct  on*  of  moro  Criticai  Owign 

Roviaw(s)  (COR)  in  aooofdanco  with  MIL*STD-1S21. 

5.4^  SoBwonolnooflnQ. 

5^1  Tho  contractor  shall  davalop  a  datafiad  dasign  for  aach  CSCI,  shall  allocata  raquiramants  from 
tha  Computar  Softwara  Componants  (CSCs)  to  tha  Computar  Softwara  UnHa  (CSUs)  of  aach  CSCI,  and 
shall  astablish  dasign  raquiramants  for  aach  CSU.  Tha  contractor  shall  documant  this  infonnatlon  in  tha 
Sollwara  Oaaign  Documant  (SOD)  for  aach  CSCI. 

5.4.2.2  Tha  contractor  shall  davalop  tha  datsilad  dasign  of  tha  CSCI  axtamal  Intarfacas  documantad  In 
tha  Intarfaca  Raquiramants  Spacification  (IRS).  Tha  contractor  shall  documant  this  infonnation  in  tha 
Intaifaca  Dasign  Documant  (IDO). 


5.4.2.3  Tha  contractor  shall  documant  any  additional  anginaaring  information  ganaratad  during  tha 
datailad  dasign  procass  in  SacUon  6  of  tha  Softwara  Dasign  Documant  (SOO)  for  aach  CSCI.  Tha 
anginaaring  information  shall  includa  rationaia,  rasults  of  analysas  and  trada<qff  studiaa,  and  any  othar 
infonnation  that  aids  in  undarstanding  tha  datailad  dasign. 

SA2.4  Tha  contractor  shall  astablish  tast  rasponsibilHiaa,  tast  caaas  (In  tarms  of  inputs,  aspactad 
rasults,  and  avaluation  criteria),  and  schaduiaa  for  CSC  integration  and  tasting.  Tha  contractop  shall 
taoord  this  information  (directly  or  by  rafaranca)  in  tha  CSC  software  davalopmant  fllos. 

S.4.2.S  Tha  contractor  shall  astablish  test  rcqulrannams,  teat  rasponsibilitlas,  teat  cases  On  tanns  of 
inputs,  axpactad  rasults.  and  evaluation  critarta),  and  schedules  for  tasting  ail  CSUs.  Tha  contractor’s 
CSU  testing  shall  includa  stressing  the  softwara  at  tha  limits  of  its  spocHtad  raquiramants.  Tha 
contractor  shall  record  this  information  (directly  or  by  rafaranca)  in  tha  CSU  softwara  davalopmant  files. 

5.4.3  Formal  ouatiflcatiQn  testing.  Tha  contractor  shall  identify  and  dascriba  tha  test  cases  for  the 
formal  qualification  test  identifications  in  tha  Softwara  Test  Plan  (STP).  Tha  contractor  shall  documant 
this  information  in  tha  Softwara  Test  Description  (STD)  for  aach  CSCI. 

5.4.4  Software  product  avaluations.  Tha  contractor  shall  perform  evaluations  of  tha  products  identified 
below,  using  tha  evaluation  criteria  specified  in  Figure  7.  Tha  contractor  shall  present  a  summary  of 
tha  evaluation  results  at  the  Critical  Design  Review(s). 

a.  Tha  updated  Software  Design  Documant  (SOD)  for  each  CSCI 

b.  The  updated  Interface  Design  Document  (IDO) 

c.  CSC  test  cases 

d.  CSU  tast  requirements  and  test  cases 

a.  A  spaciflad  percentage  of  the  set  of  CSU  and  CSC  software  development  files  (SDFs).  The 
spacifiad  percentage  shall  be  as  identified  in  the  Software  Development  Plan  (SDP). 
f.  T^  Softvi^  Tast  Description  (STO)  for  each  CSCI. 

5.4.5  Configuration  management. 

5.4.5.1  The  contractor  shall  incorporate  the  updated  Softwara  Design  Document  (SDO)  for  each  CSCI 
into  tha  CSCI's  Developmental  Configuration  prior  to  delivery  to  tha  contracting  agency. 

5.4.5.2  Tha  contractor  shall  place  the  updated  Interface  Design  Document  (IDD)  under  configuration 
corttrol  prior  to  delivery  to  the  contracting  agency. 
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5.5  Codlg 
actMUM. 


Th«  contractor  shall  pofform  tha  fbllowina  Coding  and  CSU  tasting 
maamant  No  additional  rsquiramants. 


5  5A1  Tha  contractor  shall  davalop  last  procaduraa  for  conducting  aach  CSU  tast  Tha  contractor 
shall  racofd  thasa  procaduraa  in  tha  corrasponding  CSU  softwara  davalopmant  filas  (SDFs). 

Tha  contractor  shall  coda  and  taat  aach  CSU  ansuifng  that  tha  algorithms  and  logic  ampioyad 
by  aach  CSU  aia  corract  and  that  tha  CSU  satisflas  its  spaciflad  raquiramants.  Tha  contractor  shail 
raooid  tha  tast  rasults  of  all  CSU  tasting  in  tha  corrasponding  CSU  SOFs. 

5  5.23  Tha  contractor  shall  make  all  nacassary  revisions  to  tha  dasign  documantation  and  coda, 
partomi  all  nacassary  ratasting.  and  shall  update  tha  SOFs  of  all  CSUs  that  undergo  dasign  or  coding 
changes  based  on  CSU  tests. 

533.4  Tha  contractor  shall  davalop  tast  procedures  for  conducting  aach  CSC  tost  Tha  contractor 
ahaH  record  thasa  procaduraa  in  tha  C^  SOFs. 

533  Pawnel  Qualification  testing.  No  additional  requiremarTta. 

53.4  §g({y>yfe  product  avluationa.  Tha  contractor  Shall  parfOrm  evaluations  of  tha  products  Wanliflad 
below,  using  tha  evaluation  criteria  spacifiad  in  Figure  8. 

a.  Tha  source  coda  fer  each  CSU 

b.  Tha  CSC  test  procedures 

c.  Tha  CSU  tast  procedures  and  tast  results 

d.  A  spacifiad  percentage  of  tha  sat  of  updated  software  development  files  (SDFs). 


5.5.5.1  Tha  contractor  shall  incorporate  tha  updated  Software  Dasign  Documents  (SDDs)  and  source 
coda  listings  for  aach  successfully  tasted  and  evaluated  CSU  into  tha  appropriate  Developmental 
Configuration. 

5.5.5.2  Tha  contractor  shall  place  tha  source  coda  for  each  successfully  tested  and  evaluated  CSU 
under  configuration  control. 
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5J  esc  infartten  «nd  ftina.  Th«  oonirKtef  thU  pwtefm  th«  (Wa«»lng  C8C  Int^ri^ 
acUvNiM. 


5.8.1 _ _ 

R«vi«w(«)  (TRR)  In  I 


tiwnaoatnant.  Th«  contractof  ahaA  conduct  on#  cr  mora  Taat  Raadinass 
iwlthMtL-STD-1521. 


5.8.2  SoRwafO  endnaarinQ. 

5.8.2.1  Tha  contractor  shall  conduct  CSC  IntagraUon  and  taatino.  Tha  contractor  shall  ansura  that  tha 
algonthma  and  logic  amployad  by  aach  CSC  ara  conact  and  that  tha  CSC  satiaflas  its  spacifiad 
raquiramanla. 


5.8.2.2  Tha  contractor  shall  record  tha  tast  raauHa  of  all 
oorraapendlng  CSC  soflwara  davalopmant  filas  (SOFs). 


CSC  Intagration  and  tasting  In  tha 


5.6.2.3  Tha  contractor  shall  maka  all  nacassary  rawiaiona  to  tha  design  documentation  and  coda. 
paifOrm  ail  nacassary  ratesting,  and  update  tha  software  davalopmant  files  (SOFs)  of  aH  CSUs,  CSCs,  and 
CSCis  that  undergo  design  or  coding  changes  based  on  tha  results  of  aB  tasting  padbrmad. 

5.8J  Formal  Qualification  testing. 

5.8J.1  For  aach  formal  qualification  tast  case  idantHlad  in  tha  Software  Taat  Daacription(s)  (STDs)  the 
contractor  shall  develop  sat-up  procedures,  procedures  tar  conducting  each  test,  and  proerKturaa  for 
analyzing  the  tast  results.  These  procedures  shall  be  documented  in  tha  Software  Tost  Description  (STD) 
tar  each  CSCI. 

5.841.2  Prior  to  conducting  Formal  Qualification  Tasting  (FQT),  tha  contractor  shall  conduct  the  tests 
documented  in  the  Software  Test  Description  (STD)  for  each  CSCI  to  ansura  that  tha  procedures  ara 
complete  and  accurate  and  that  the  soilwara  is  ready  for  FQT.  Tha  contractor  shall  record  the  resuifo 
of  this  activity  In  the  corresponding  CSCI  software  development  files  (SOFs)  and  shall  update  the  STD 
as  appropriate. 

5.6.4  Software  product  evatuatlcna.  The  contractor  shall  perform  evaluations  of  tha  products  identified 
below,  using  the  evaluation  criteria  specified  in  Rgure  9.  Tha  contrsetor  shall  present  a  summary  of 
the  evaluation  results  at  the  Test  Readiness  Review. 

a.  The  test  results  recorded  in  the  software  development  files  (SOFs) 

b.  The  updated  Software  Test  Description  (STD)  for  each  CSCI 

c.  Tha  updated  source  coda  and  da^gn  documents 

d.  A  specified  percentage  of  tha  updated  software  development  files  (SOFs). 


>5, 
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5.8.5  Conflouratjon  management.  Tha  contractor  shall  incorporate  the  updated  Software  Designs 
Documents  (SOOs)  and  source  code  listings  for  each  successfully  tested  and  evaluated  CSC  into  ther|^ 
appropriate  Developmental  Configuration.  ^ 
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5.7  Th«  contf«ctof  *h*l»  P«»term  th#  following  CSCI  tMting  activltios. 


5.7.1  dovomnmont  manaotmiOt-  ConflgufiWoo 

(PGA)  MKt  Physieai  Configuratloo  AudM(a)  (PCA).  Tho  FCA  and  PCA  for  a  CSCI  may  bo  dolayod 

until  aftor  syatom  integration  and  tasting.  • 


5.7.2.1  Tho  contractor  shall  mako  nocossary  rovisiona  to  tho  Soflwaro  Dosign  Oocumont(s)  (SOOa)  and 
coda,  conduct  all  nocasaa/y  ratosting,  and  update  tho  softwara  davalopmant  filaa  (SOFs)  of  ail  CSUs, 
CSCs,  and  CSCIs  that  undargo  dosign  or  coding  changoa  basad  on  tha  rasutts  of  formal  qualification 
tasting. 


5.7.2L2  Tha  contractor  shall  maka  nacassary  ravisions  to  tha  Intarfaca  Oasign  Oocumant  (IDO)  basad  on 


tha  rasults  of  fOnnal  qualification  tasting  and  shall  praparo  tho  iOO  for  dolivary. 


9.7, 2,3  Following  succossful  complation  of  formal  qualification  tasting  and  prior  to  Functional 
Configuratian  Audit  (FCA)  and  Physical  Configuration  Audit  (PCA).  tha  contractor  shall  produca  tha 
updated  sourca  coda  for  aach  CSCI.  Tha  contractor  shall  praparo  tha  sourca  coda  for  aach  CSCI  for{^^ 
dalivory  as  spadfiad  in  tha  Softwara  Raquiramartts  Spadflcatlon  (SRS). 


5.7.2.4  For  aach  CSCI,  tha  contractor  shail  praparo  a  Softwara  Product  Spadflcatlon  (SPS)  for;^. 
dolivary. 


5.7.3.1  Tha  contrador  shall  perform  tha  fonnal  qualification  tasting  (FQT)  acdviUaa  in  accordanca 
with  tha  procaduras  documanted  in  tha  Softwara  Tast  Dascription  (STD)  for  aach  CSCI. 


5.7.3.2  Tha  contractor  shail  record  tha  rasults  of  formal  qualification  tasting  in  tha  Softwara  Test 
Raport(s)  (STRs)  for  aach  CSCI. 


5.7.3.3  Upon  complation  of  FQT,  tha  contractor  shall  prepare  an  up-to-date  Softwara  Test  Descriptions^ 
(STD)  for  delivery  to  tha  contracting  agency. 


5,7.4  Software  product  evaluations.  Tha  contractor  shall  perform  evaluations  of  the  products  idantifiec^ 
below,  using  tha  evaluation  criteria  specified  in  Figure  1 0. 


a.  The  Soflwaro  Tast  Report(s)  (STRs)  for  each  CSCI 

b.  The  updated  sourca  code  and  design  documentation. 


5. 7.5.1  Tha  contrador  shall  identify  tha  exsd  version  of  aach  CSCI  to  be  dalivered.  Tha  contractor 

shall  document  this  information  in  a  Version  Description  Document  (VDD)  for  each  CSCI. 


5.7.5.2  Following  successful  completion  of  tha  Fundionai  Configuration  Audit  (FCA)  and  Physical^ 
Configuration  Audit  (PCA)  and  when  authenticated  by  tha  contrading  agency,  tha  Software  Product 
Specification  (SPS)  for  each  CSCI  will  be  incorporated  into  the  Produd  Baseline.  At  this  point  eact;r 
CSCI’s  Developmental  Configuration  shail  cease  to  exist 
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5^1  dawirinnfTf"*  managatTlMH.  m*  conu»«j»  .nw.  --vkw*  rr.yw«« 

Conflguntion  Autfli  (SM  5.7.1). 

.■2  •nainaarina.  Th« contrtdof  sh*ll  mtk#  n«cM8aiy  r»vtetons  to dwign  documentition  tnd 

codo  ond  aholl  poifomi  alt  rataating  nacaaaaiy  baaad  on  ayatam  Intagration  and  taatlng. 


5.&3.1  Tha  contractor  ahall  aupport  tha  davalopmant  and  documantation  at  ayatam  Intagration  and  taat 
pteia,  taat  caaaa,  and  taat  procaduraa. 


5  g  a  Tha  contractor  ahall  aupport  ayatam  intagration  and  taatlng  acthritfaa. 

5  8JJ  Tha  contractor  ahall  aupport  post  taat  analyaia  and  raportlng  of  ayatam  Intagration  and  tost 
raaulta. 

5  0.4  SnOwars  pmrfufft  as/aluatlons.  Tha  contractor  ahall  pofform  avaluatlOns  of  tha  updated  aourca  cods 

and  design  documantation  using  tha  evaluation  crttarta  spadflad  in  Rgura  10. 

5.8.5  finnaoursHnn  manacemaot  The  contractor  shaU  piapara  nacassary  changae  to  baaaBnad 
documantation  in  accordance  with  paragraph  4.5.5. 
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8.  NOTES 


8.1  fw.  il«i  and  cfo*«  rafafflca.  thi«  standard  la  uaad  In  an  aequiaition  which 

inooiporataa  a  tX>  Foim  1423.  Contract  Data  Raquiramanta  Uat  (CDRL),  tha  data  raquiramanta  idantHlad 
stwH  ba  davalopad  aa  spaciflad  by  an  approvad  Data  Itam  Oaacription  (00  Form  1664)  and 
ijjftyyy)  [fi  accordanca  with  tha  approvad  CDRL  incorporated  into  tha  contract  Whan  the  provisions  of 
tha  000  FAR  Suppiamant  27.4104  are  invoked  and  tha  00  Form  1423  ia  not  uaad,  tha  data  spaciflad 
below  shall  be  dallvarad  by  tha  contractor  in  accordanca  with  tha  contract  or  purchase  order 
raquiramanta. 


5.1  5.1.4, 

5u1.S 


Syatam/Sagmant  Oaaign 
Document  (SSOO) 


OI-CMAN-XXXXX 


4.1J,  4.2.5, 

4X1.  5.1.4,  5.1.5. 
5X4,  40.1,  40.2.5 


Software  Oavalopmant  Plan 
(SOP) 


OI-MCCR-80030 


4X8,  4X10.  4JX 
5.1X3,  5.1X  5.1.4, 
5.1.5.  5.2.1.  5.2.2.I. 
5X3,  5.2.4,  5.2.5. 
5Xi1.  5JJ,  5.7.2X 
40X1 


Software  Raquiramanta 
Specification  (SRS) 


OMMccR-aooas 


4X6,  4X4.  5. 1.2.4, 
5.1.4,  5.1.5,  5.2.1, 
5X2.2,  5.2.4,  5.2.5, 
5X2.1,  5X2.2,  5.4.2.2 


Interface  Raquiramants 
Specification  (IRS) 


DI-MCCR-80026 


5J.2.2.  5.3.4,  5.3.5.3. 
5.4.2.2.  5.4.4,  5.4.5.2, 
5.7.2.2 


Interface  Design  Document 
(IDD) 


DI-MCCR-80027 


5X2.1,  5X2X 
5J.4,  5X5.1,  5.4.2.1 
5.4.2.3,  5.4.4,  5.4.5.1 


Software  Design 
Document  (SOD) 


DI-MCCR-80012 


4.2.10,  5.7.2.4.  5.7.5.2 


Software  Product  Specification 
(SPS) 


DI-MCCR-80029 


5.7.5.1 


Version  Description  Document 
(VOO) 


DI-MCCR-a00l3 
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4^1.  13J.  SiSA 
S^SA  SAS 

4J.A  SA3.  5A4, 
5A5A  S.6J.1,S.6J.2, 
5.6A  S.7J.1.  5.733 


Data  R4QulfmMi««TWI> 

So<bMf*TMtP1an 

(STF) 

Softwar*  TMt  Description 
(STD) 


0I44CCR^14 

0I44CCR^15 


5.733,  5.7.4 

4.6A 


Software  Tost  Rapoft 
(STR) 


Computer  System  Operator's 
Manual  (CSOM) 


DWMCCR-60017 

OI-MCCR-80018  ^ 


4.6.4 

4.6.4 

4.6.4 


Software  User's  Manual  (SUM) 

Software  Programmer's  Manual 
(SPM) 

Firmware  Support  Manual  (PSM) 


OKMCCR^ig 

OM6CCR-80021 

: 

OMMCCR-60022 


i 


4.6.2,  4.6.4  Computer  Resources  Integrated  OI44CCR-60024  M 

Support  Document  (CRISO) 


(Data  item  descriptions  related  to  this  standard,  and  identified  in  section  6  will  be  approved  and  listed  ' 
as  such  in  OoO  5000. 19-L,  Vol.  II.  AMSOL  Copies  of  data  item  descriptions  required  by  the  contractors 
in  connection  with  specific  acquisition  functions  shouid  be  obtained  from  the  Naval  Publications  and  S 
Forms  Center  or  as  directed  by  the  contracting  officer.)  ^ 


8.2  Subject  term  (kev  wordi  listing. 

Acquisition 

Baselines 

Code 

Coding  and  CSU  testing 
Computer 

Computer  resources 
Computer  software 
Computer  software  component 


Computer  software  configuration  item 

V 

Computer  software  unit 

Configuration  item 

Configuration  management 

Sl 

CSC 

M 

CSC  irttegration  and  testing 

h  • 


§ 
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CSCI 

CSCItMtInfl 
CSU  _ 

Dilt  tlwn 
DataiM  dMign 
Ocvatopmantal  configuration 
Enginaaring  anvironmant 
Fimiwara 

Formtf  Qualification  Tasting 
Non^avalopmantal  softwara 
praHminaiy  daaign 
OualHIcation 
Raquiromants  analysis 
Risk  managamant 
Safaty  managamant 
SoftMtara 

Softwara  davalopmant 
Softwara  davalopmant  fila 
Softwara  davalopmant  library 
Softwara  anginaarfng 

Softwara  product  avaluation 

Softwara  raquiramants  analysis 

Softwara  support 

Systam  intagration  and  tasting 

Tast  anvironmant 

Tasting 
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UST  OF  ACRONYMS  AND  ABBREVIATIONS 


10.1  This  appendix  provides  a  list  of  all  acronyms  and  abbreviations  used  in  this  standard. 

wMi  the  associated  meaning. 


COR 

Critical  Design  Review 

CORL 

Contract  Data  Requirements  List 

CIOS 

CrMcai  Item  Development  Specification 

CRISO 

Computer  Resources  Integrated  Support  Document 

CSC 

Computer  Software  Component 

CSCI 

Computer  Software  Configuration  Item 

C80M 

Computer  System  Operator's  Manual 

CSU 

Computer  Software  Unit 

DIO 

Data  Item  Description 

000 

Department  of  Defense 

OOOISS 

Dapaitmant  of  Defense  Index  of  Spedflcations  and  Standards 

ECP 

Engineering  Change  Proposai 

FAR 

Federal  Acquisition  Regulation 

FCA 

Functional  Configuration  Audit 

FSM 

Rrmware  Support  Manual 

FQT 

Formal  Qualification  Testing 

GFS 

Government  Furnished  Software 

HOL 

High  order  language 

HWCI 

Hardware  Configuration  Kern 

100 

Interface  Design  Document 

I/O 

Input/Output 

IRS 

Interface  Requirements  Specification 

IV4V 

Independent  Verification  and  Validation 

NOS 

NorKlevelopmental  software 

PCA 

Physical  Configuration  Audit 

FOR 

Preliminary  Design  Review 

PIGS 

Prime  Item  Development  Specification 

SCN 

Specification  Change  Notice 

SOO 

Software  Design  Document 

SOF 

Software  development  file 

SOL 

Software  development  library 

SOP 

Software  Development  Plan 

SOR 

System  Design  Review 

SOW 

Statement  of  WorV 

SPM 

Software  Programmer's  Manual 

SPS 

Software  Product  Specification 

SRR 

System  Requirements  Review 

SRS 

Software  Requirements  Specification 

SSOO 

System/Segment  Design  Document 

SSR 

Software  Specification  Review 

SSS 

System/Segment  Specification 

STD 

Software  Test  Description 

STP 

Software  Test  Plan 

STR 

Software  Test  Report 
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APPENDIX  B 

REOUIREMENTS  FOR  SOFTWARE  CODING  STANDARDS 


20.1  Pufpoaa.  Th«  purpoM  of  this  appondix  is  to  spscify  languago  indspsndent  rsquirsmants  for 
softwsrs  coding  standards. 

20.2  AaoilfabHHv.  This  appondix  appllas  to  ail  dslivsrsbis  sourcs  cod*  dovsiopsd  undsr  the  contract 

203  Rules  and  Conventions.  The  following  subparagraphs  define  the  requirements  for  rules  and 
conventions  applicable  to  software  coding  standards.  The  contractor  shall  implement  software  coding 
standards  that  comply  with  these  requirements. 

203.1  Preaentafion  style.  The  coding  standards  shal!  describe  the  rules  and  conventions  for  the  format 
of  the  source  code  which  may  include  paper  listings,  listings  stored  on  electronic  media,  or  both.  The 
njiee  and  conventions  for  presentation  stylo  shall  include  standards  fon 

a.  indentafion  and  spacing 
tai  The  use  of  capKaiization 

a  Uniform  presentation  of  information  throughout  the  source  code  (e.g.«  the  grouping  together  of 
all  data  declarations) 

d.  Use  of  headers 

e.  Layout  of  source  code  listings 

f.  Conditions  under  which  comments  are  provided  and  the  format  to  be  used 

g.  Size  of  code  aggregates  (e.g.  on  the  average  100  or  at  most  200  executabis,  non-expandable 
statements). 

2033  Naming.  The  coding  standards  shall  describe  the  rules  and  conventions  governing  the  selection  of 
iderttHlers  used  in  the  source  code  listings  (eig.,  identifiers  for  CSUs,  variables,  parameters,  packages, 
procedures,  subunits,  and  other  aggregates  of  code.)  Restrictions  on  the  use  of  reserve  words  and 
keywords  shall  be  identiried. 

20.3.3  Restrictions  on  the  implementation  language.  The  coding  standards  shall  include  a  description  of 
any  restrictions  imposed  on  the  use  of  constructs  and  features  of  the  implementation  language  due  to 
project  or  machine-dependent  characteristics.  MachineKlependent  characteristics  may  include 
input/output  features,  word  length-dependent  features,  use  of  floating  point  arithmetic,  etc.  Project 
characteristics  may  include,  but  are  not  limited  to,  safety  or  security  considerations  in  the  operational 
environment. 

20.3.4  Use  of  language  constructs  and  features.  The  coding  standards  shall  address  the  allowed  use  of 
constructs  and  features  of  the  implementation  language.  For  example,  when  Ada  is  the  implementation 
language,  the  coding  standards  shall  address  the  use  of  exception  handling,  goto  and  abort  statements, 
unchecked  conversion,  etc. 

203.S  Complexity.  The  coding  standards  shall  describe  controls  and  restrictions  on  the  complexity  of 
code  aggregates. 
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appendix  C 

CATEGORY  AND  PRIORITY  CLASSIFICATIONS 
FOR  SOFTWARE  PROBLEM  REPORTING 


30 1  Pumosa  This  appendix  contains  a  catsgo»y  and  priority  dassiflcstion  schsm*  to  bo  appliod  to  all 
p^btoms^octed  in  tho  dollvorablo  softwaro  tha«  has  boon  placod  undor  oontractof  configuration 

control. 


30.2  CliipificatlQn  t 
catogory  as  follows: 


y.  Probloms  dotoctod  during  softwars  operation  shall  bo  dassiflod  by 


a.  Snfiwre  orobiem.  Tho  softwaro  does  not  oporato  according  to  supporting  documentation  and 
tho  documentation  is  correct 

jx  OoeumentaWan  orciblam.  Tho  softwaro  does  not  Operate  according  to  supporting  documentation 
but  tho  software  operation  is  correct 

c.  t\— Mfi  nfnMem.  The  softwaro  operated  according  to  supporting  documentaBon  but  a  design 
doAdoncy  exists.  Tho  design  deficiency  may  not  always  result  in  a  dlrodiy  obsorvablo 
operational  symptom  but  possesses  the  poterrtlai  for  creating  further  problems. 


30  J  211 
follows: 


y.  Problems  detected  In  the  software  shall  be  dassHled  by  prlorty  as 


a.  Priority  1.  A  software  problem  that  does  one  of  the  following; 

(1)  Prevents  the  accomplishment  of  an  operational  or  mission  essential  capability  specified  by 
baselined  requirements 

(2)  Prevents  the  operator’s  accomplishment  of  an  operational  or  mission  essential  capability 

(3)  Jeopardizes  personnel  safety. 

b.  PrioriN  2.  A  software  problem  that  does  one  of  the  following: 

(1)  Adversely  affects  the  accomplishment  of  an  operational  or  mission  essential  capability 
specified  by  baselined  requirements  so  as  to  degrade  performance  and  for  which  no 
alternative  worV-around  solution  is  known 

(2)  Adversely  affects  the  operator's  accomplishment  of  an  operational  or  mission  essential 
capability  specified  by  baselined  requirements  so  as  to  degrade  performance  and  for  which 
no  alternative  work-around  solution  is  known. 

c.  Priority  3.  A  software  problem  that  does  one  of  the  following: 

(1)  Adversely  affects  the  accomplishment  of  an  operational  or  mission  essential  capability 
specified  by  baselined  requirements  so  as  to  degrade  performance  and  for  which  an 
alternative  work-around  solution  is  known 

(2)  Adversely  affects  the  operator’s  accomplishment  of  an  operational  or  mission  essential 
capability  specified  by  baselined  requirements  so  as  to  degrade  performance  and  for  which 
an  alternative  work-around  solution  is  known. 
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APPENOCXO 
EVALUATION  CRfTERIA 

40.1  PuHMMa.  This  appendix  contains  a  default  set  of  deAnitlona  for  the  evaiuation  criteria  appearing  in 
Figures  4  through  10.  These  definitiona  shall  be  implemented  by  the  contractor  if  an  alternative  set  has 
not  been  proposed  In  the  Software  Development  Plan  and  accepted  by  the  contracting  agency. 

40.2  Criteria  daUntMons.  The  following  deflnttlons  are  listed  in  the  order  that  the  criteria  appear  in 
Figures  4  through  10.  For  convenience,  the  definitiona  use  the  word  'document*  for  the  item  being 
evahiated,  even  though  in  some  instances  the  Item  being  evaluated  may  be  other  than  a  document 

40JL1  Internal  consistency.  Internal  consistency  as  used  in  this  standard  means  that  (1)  no  two 
statemerrls  In  a  document  contradict  one  another.  (2)  a  given  term,  acronym,  or  abbreviation  means  the 
same  thing  throughout  the  document  and  (3)  a  given  item  or  concept  Is  referred  to  by  the  same  name 
or  description  throughout  the  document 

40.2.2  Understandabilltv.  Understandability,  as  used  In  this  standard,  means  thab  (1)  the  document  uses 
niies  of  grammar,  capitalization,  punctuation,  symbols,  and  notation  eorraistent  with  those  specified  in 
the  U.  S.  Government  Printing  Office  Style  ManuaL  (2)  ail  terms  not  contained  in  the  U.  S.  Government 
PiMing  Office  Style  Manual  or  MerriarrvWebster's  New  Intemaborrai  dlctiorrary  (latest  revision)  are 
defined.  P)  standard  abbreviations  listed  in  MIL'STD>12  are  used,  (4)  all  acronyms  and  abbraviatioao  not 
listod  In  MiL<STD>l2  are  defined,  (S)  all  acrorryma  and  abbrevlailom  are  preceded  by  the  word  or  term 
speited  out  in  hill  the  first  time  they  are  used  in  the  document  unless  the  llrat  use  occurs  in  a  table, 
figure,  or  equation,  in  which  case  they  are  erqplalned  in  the  taad  or  in  a  footnote,  and  (8)  all  tables, 
figures,  and  illustrationa  are  called  out  in  the  text  before  they  appear,  in  the  order  in  which  they 
appear  in  the  document 

40.2P  Traceabilitv  to  indicated  documents.  Traceability  as  used  in  this  standard  means  that  the 
document  in  question  is  in  agreement  with  a  predecessor  document  to  which  it  has  a  hierarchical 
raiationship.  Traceability  has  five  elements:  (1)  the  documerrt  in  question  corrtains  or  implements  ail 
applicabie  stipulations  cif  the  predecessor  document  (2)  a  given  tenm,  acronym,  or  abbreviation  means  the 
same  thing  in  the  documents,  (3)  a  given  item  or  concept  is  referred  to  by  the  same  name  or  description 
in  the  documents,  (4)  all  material  in  the  successor  document  has  its  basis  in  the  predecessor  document 
that  Is,  no  untraceable  material  has  been  ir\troducsd,  and  (5)  the  two  documents  do  not  contradict  one 
another. 

40.2.4  Consistency  with  indicated  documents.  Consistency  between  documents,  as  used  in  this  standard, 
means  that  two  or  more  documents  that  are  not  hierarchically  related  are  free  from  contradictions  with 
one  another.  Elements  of  consistency  are:  (1)  no  two  statements  contradict  one  another,  (2)  a  given 
term,  acronym,  or  abbreviation  means  the  same  thing  in  the  documents,  and  (3)  a  given  item  or  concept 
is  referred  to  by  the  same  name  or  description  in  the  documents. 

40.2.5  Aoofooriate  analysis,  design,  and  coding  techniques  used.  The  contract  may  include  provisions 
regarding  the  requirements  analysis,  design,  and  coding  techniques  to  be  used.  The  contractor's 
Software  Development  Plan  (SOP)  describes  the  contractor's  proposed  implementation  of  these  techniques. 
This  criterion  consists  of  compliance  with  the  techniques  specified  in  the  contract  and  SOP. 

40.2.6  Aporooriate  allocation  of  sizing  and  timing  resources.  This  criterion,  as  used  in  this  standard, 
means  that  (1)  the  amount  of  memory  or  time  allocated  to  a  given  element  does  not  exceed  documented 
constraints  applicable  to  that  element,  and  (2)  the  sum  of  the  allocated  amounts  for  all  subordinate 
eiements  is  within  the  overall  allocation  for  an  item. 


45 


. . 


000«TD>2ie7A  (OfUFT) 
Appendix  0 


40^7  «— » r««. .ir«mxn«a.  Thit  cdtadon.  ••  used  in  this  standard,  means  that  m) 

every  apedfled  rsquiiement  is  addressed  by  at  least  one  test,  (2)  test  cases  have  been  seisetsd  tor  bi^ 
*aversge*  situation  and  *boundary*  situatiorts,  such  as  minimum  and  maximum  values,  (3)  ’stress*  cases 
have  been  selected,  such  as  out<««ounds  values,  and  (4)  test  cases  that  exercise  combinationx  of 
different  functions  are  indudad. 


40J  AtMittenil  glta tit.  The  tallowing  definitions  apply  to  edterta  that  are  not  seff-expianatory  md 
that  appear  in  the  NOTES  column  of  Rgures  4  through  10.  These  ertterta  ate  not  included  in  each 
figure,  but  appear  only  aa  appropriate. 


4W.t  A^ICY  gf  fluallttf  factora.  This  ertterton  applies  to  the  quality  factor  requiiementa  in  the 
Software  Requirements  Specification  (SRS).  Aapects  to  be  considered  are:  (1)  tredeefla  between  quaiitv 
fa^  hasm  been  considered  and  documented,  and  (2)  each  quality  factor  accompanied  by  a  feasible 
method  to  be  used  to  evaluate  compliance  with  it  is  requir^  by  the  SRS  OiO. 


Iffgtabi«ttv  Of-aq.tii>B>?flnta.  a  requirement  is  considered  to  be  testable  if  an  oblectfve  and 
faasibie  teat  can  be  designed  to  determine  whether  the  requirement  is  met  by  the  software. 


^•3-3  COMiypcv  between  data  defInttiQn  and  dar^  This  criterion  applies  primarily  to  deaion 
demerits.  It  means  that  each  data  element  is  defined  In  a  way  that  is  eor>aiatant  with  Hs  usage  in  the 


Bggy urwjtffat  Inotif.  emected  results,  evafuatfon  erftortai.  Test 
cases  and  test  procedures  should  specify  exactly  what  inputs  to  provide,  what  steps  to  fellow,  what 
outpiJta  to  expect,  and  what  criteria  to  uae  In  evaluating  the  outputs.  If  any  of  these  elements  are  not 
specified,  the  test  case  or  test  procedure  is  inadequate. 


^;3.5  CgtralfftfinttM  qf  toathq.  Testing  is  complete  W  all  iest  cases  and  all  test  procedures  have  been  a 
perfermed,  all  results  have  been  recorded,  and  ail  acceptance  criteria  have  been  met 


40.3.6  CflPBlfftffnqgS  qf  retesting-  Retesting  consists  of  repeating  a  subset  of  the  test  cases  and  test 
pr^dures  after  software  corrections  have  been  made  to  correct  problems  found  in  previous  testing, 
Retesting  is  considered  complete  if:  (1)  ail  test  cases  and  test  procedures  that  revealed  problems  in  the 
previous  testing  have  been  repeated,  their  results  have  been  recorded,  and  the  results  have  met  • 
acceptance  criteria,  and  (2)  all  test  cases  and  test  procedures  that  revealed  no  problems  during  the 
previous  testing,  but  that  test  functions  that  are  affected  by  the  corrections,  have  been  repeated,  their 
results  b06n  rscordsd*  and  tha  rasutts  ^'av•  met  acceptance  criteria* 
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